ulysses/scripts
Sirsyorrz 9955f5bc95
Fix VRAM estimates for pre-quantized HF repos
The Cookbook fit scanner was reporting impossibly low VRAM requirements
for some pre-quantized models — e.g. cyankiwi/Qwen3-Coder-Next-REAM-AWQ-4bit
shown as 7.1 GB ('perfect' on a 12 GB card) when the real load is ~40 GB.

Root cause is in the catalog builder. When _entry_from_modelinfo falls
back to safetensors metadata for the parameter count, it stored
safetensors.total directly. For pre-quantized repos that figure reflects
*packed* element counts: AWQ/GPTQ-Int4 pack 8x 4-bit weights into one
I32, AWQ-8bit/GPTQ-Int8/FP8 pack 4x. The catalog therefore recorded
~1/8 of the real parameter count, and min_vram_gb = packed * bpp
double-applied the quantization.

Fix the safetensors fallback:

* prefer the per-dtype parameters dict when available and unpack only the
  I32/I64 entries (the F16/BF16 scale/zero tensors and embeddings are
  already at their real element counts)
* fall back to total * pack_factor when only total is exposed

Patch the catalog entries that were affected by the old fallback so the
fit ratings reflect reality without waiting for a full catalog rebuild:

* cyankiwi/Qwen3-Coder-Next-REAM-AWQ-4bit  11.4B -> 79.7B (40.8 GB VRAM)
* stelterlab/Qwen3-Coder-30B-A3B-Instruct-AWQ  4.6B -> 30.5B
* stelterlab/NVIDIA-Nemotron-3-Nano-30B-A3B-AWQ  5.1B -> 30.5B
* warshanks/Qwen3-8B-abliterated-AWQ  2.2B -> 8.2B
* QuantTrio/sarvam-30b-AWQ  7B -> 30B
* QuantTrio/sarvam-105b-AWQ  19B -> 105B

Closes #377.
2026-06-01 18:32:58 +09:00
..
_completion Odysseus v1.0 2026-05-31 23:58:26 +09:00
_lib Odysseus v1.0 2026-05-31 23:58:26 +09:00
demo_email Odysseus v1.0 2026-05-31 23:58:26 +09:00
add_hwfit_models.py Fix VRAM estimates for pre-quantized HF repos 2026-06-01 18:32:58 +09:00
claim_ownerless.py Add native Windows compatibility layer 2026-06-01 15:09:47 +09:00
diffusion_server.py Add native Windows compatibility layer 2026-06-01 15:09:47 +09:00
encode_previews.sh Odysseus v1.0 2026-05-31 23:58:26 +09:00
fix_paths.py Odysseus v1.0 2026-05-31 23:58:26 +09:00
hf_download.py Odysseus v1.0 2026-05-31 23:58:26 +09:00
index_documents.py Odysseus v1.0 2026-05-31 23:58:26 +09:00
migrate_faiss_to_chroma.py Add native Windows compatibility layer 2026-06-01 15:09:47 +09:00
odysseus Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-backup Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-calendar Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-contacts Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-cookbook Add macOS Apple Silicon Cookbook support 2026-06-01 14:59:19 +09:00
odysseus-docs Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-gallery Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-logs Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-mail Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-mcp Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-memory Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-notes Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-personal Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-preset Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-research Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-sessions Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-signature Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-skills Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-tasks Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-theme Odysseus v1.0 2026-05-31 23:58:26 +09:00
odysseus-webhook Odysseus v1.0 2026-05-31 23:58:26 +09:00
update_database.py Odysseus v1.0 2026-05-31 23:58:26 +09:00