Skip to content

v3.18.0 — Local model optimization + self-update

Choose a tag to compare

View all tags
@eniz1806 eniz1806 released this 27 Jun 17:01
v3.18.0
1da7eb2

Make local models smaller to download, faster to run, and lighter on RAM — and let Kodiqa keep itself up to date. Backed by docs/model-optimization-research.md.

Added — local model optimization

  • Flash attention + KV-cache quantization on by default for Kodiqa-spawned Ollama servers (OLLAMA_FLASH_ATTENTION=1, OLLAMA_KV_CACHE_TYPE=q8_0) — ~half the KV-cache RAM, faster long contexts, negligible quality loss. Tune via config flash_attention / kv_cache_type; your own OLLAMA_* env vars win.
  • "Fits my machine" — detects your VRAM/RAM budget and annotates the model list + HuggingFace quant picker with ✓ fits / ⚠ tight / ✗ too big; flags sub-4-bit quants as low-bit (may hurt coding).
  • Better quant sourcing — the HuggingFace fallback prefers imatrix/dynamic uploaders (Unsloth UD-, bartowski) and labels them.
  • /tune — set local runtime knobs (num_ctx, KV-cache type, flash attention, GPU layers).
  • /quantize — build a smaller variant on-device via ollama create --quantize.
  • /recommend — strong local coding models (Qwen-Coder + a REAP-pruned MoE), with size + fit.

Added — self-update

  • Startup PyPI check (once/day, offline-safe) notifies when a newer Kodiqa release exists; new /upgrade runs pip install -U kodiqa. Disable via check_app_update: false.

Fixed

  • save_settings no longer silently drops API keys on a partial save (preserves on-disk *_api_key values unless explicitly cleared) and keeps a settings.json.bak.

566 tests pass, ruff clean.

🤖 Generated with Claude Code

Assets 2
Loading