Allow serve-profile generation for models whose trained context window is below 8192 while preserving the 8K shrink floor for larger models. |
||
|---|---|---|
| .. | ||
| data | ||
| __init__.py | ||
| fit.py | ||
| hardware.py | ||
| image_models.py | ||
| models.py | ||
| profiles.py | ||
Allow serve-profile generation for models whose trained context window is below 8192 while preserving the 8K shrink floor for larger models. |
||
|---|---|---|
| .. | ||
| data | ||
| __init__.py | ||
| fit.py | ||
| hardware.py | ||
| image_models.py | ||
| models.py | ||
| profiles.py | ||