Repository navigation
v9.3.0 - the finale
Release Notes
This release brings a major refresh of the model lineup, a rebuilt Ask Jeeves assistant, a new MiniMax backend (credit to @octo-patch), and a large round of dependency upgrades and stability fixes. It's likely the last release before upgrading to transformers 5+, which will enable additional features but may deprecate certain models.
🤖 Chat Models
New models
| Model | Details |
|---|---|
| Phi 4 – 14b | ➕ Excels at not hallucinating, essential for RAG |
| Gemma 3 – 4b | ➕ Excels at not hallucinating, essential for RAG |
| Gemma 3 – 12b | ➕ Excels at not hallucinating, essential for RAG |
| Granite – 3b | 🔄 Upgraded from Granite 3.3 → 4.1 |
| Granite – 8b | 🔄 Upgraded from Granite 3.3 → 4.1 |
| LiquidAI – 1.2b | 🔄 Updated to LFM2.5-1.2B-Instruct |
- Qwen 3 models now run in non-thinking mode for direct, concise answers. Research shows that "thinking" appreciably reduces quality for purposes of RAG.
| Removed Models |
|---|
| Deepseek R1 – 8b |
| GLM4-Z1 – 9b |
| GLM4-Z1 – 32b |
| Seed Coder – 8b |
| EXAONE |
| Phi 4 Mini – 4b |
| Qwen 3 – 4b (Thinking) |
| Qwen 3 – 32b |
- All removed models are either too heavy for too low quality compared to newer models or require "thinking" (or both).
🧬 Embedding Models
New models
| Model | Details |
|---|---|
| harrier-oss-v1-270m | ➕ Great generalist and also excels at multilingual (~94 languages) |
| harrier-oss-v1-0.6b | ➕ Great generalist and also excels at multilingual (~94 languages) |
| Octen-Embedding-0.6B | ➕ Great generalist and also excels at legal, medical, finance, and code. |
| Octen-Embedding-4B | ➕Great generalist and also excels at legal, medical, finance, and code. |
| Octen-Embedding-8B | ➕ Great generalist and also excels at legal, medical, finance, and code. |
| modernbert-embed-base_finetune_512 | ➕ Excels at legal. |
| modernbert-embed-base_finetune_8192 | ➕ Excels at legal |
| EmbeddingGemma | ➕ Google, 300M — Strong and efficient generalist |
| Removed Model |
|---|
| Granite-30m-English |
| Granite-125m-English |
| inf-retriever-v1-1.5b |
| inf-retriever-v1-7b |
| arctic-embed-m-v2.0 |
| arctic-embed-l-v2.0 |
- The removed models were essentially superseded by smaller models with at-par or better results.
👁️ Vision Models
New / updated models
| Model | Details |
|---|---|
| Liquid-VL – 480M | ➕ New — LFM2-VL family, fast and low-VRAM |
| Liquid-VL – 1.6B | ➕ New — LFM2-VL family, fast and low-VRAM |
| Liquid-VL – 3B | ➕ New — LFM2-VL family, fast and low-VRAM |
| Qwen3-VL – 2B | ➕ Qwen VL lineup (Qwen3-VL generation) |
| Qwen2.5-VL – 3B | ➕ Qwen VL lineup |
| Qwen3-VL – 4B | ➕ Qwen VL lineup (Qwen3-VL generation) |
| Qwen2.5-VL – 7B | ➕ Qwen VL lineup |
| Granite Vision 3.2 – 2B | 🔄 Updated to 3.2 (2B) |
| InternVL3 – 1B | 🔄 Now loads from the official -HF repo |
| InternVL3 – 2B | 🔄 Now loads from the official -HF repo |
| InternVL3 – 8B | 🔄 Now loads from the official -HF repo |
Performance
- Qwen-VL patch-embed
Conv3dis now swapped for a mathematically identical matmul, sidestepping a slow cuDNN kernel that could cost ~12–15s per image on some GPUs.
| Removed Model |
|---|
| Ovis2 |
| Florence-2 |
| GLM-4.1V-9B-Thinking |
| InternVL3 – 14b |
| Molmo-D-0924 |
- All superseded by newer models that are lighter and/or produce better results.
🔊 Text-to-Speech
- Added Kyutai (GPU) and Kyutai Pocket (CPU) backends.
- Chatterbox can now run on CPU or GPU.
☁️ MiniMax Backend
- Added MiniMax-M3 chat backend.
🛎️ Ask Jeeves
- Rebuilt from the ground up — Jeeves now runs the same lightweight, unquantized
CHAT_MODELS(LiquidAI and Qwen 3 small models) instead of the old CTranslate2 int8 models. - New Sources panel — a collapsible "Show sources" view lists the retrieved user-guide chunks with clickable links and similarity scores.
- Refined the butler persona for clearer, more in-character answers.
- Added an easter egg...
🖥️ UI & Features
- Local chat models now work on CPU — the GPU-only restriction has been removed.
- The GPU comparison chart plus backup/restore remain; the chat/vision/vector model comparison charts have been retired.
- Expanded user manual coverage.
🐛 Stability & Bug Fixes
- Pinned numpy to 2.3.4 to avoid a heap-corruption regression in numpy 2.4.x during large TileDB builds, and added a
np.in1d → np.isinshim for tiledb-vector-search. - Generation threads now propagate errors instead of failing silently; metrics-bar collection moved to an interruptible loop.
- Centralized subprocess output draining with proper timeout handling so stuck stages can't hang the app.
- Embedding pipeline now safely drops and reports chunks that fail tokenization rather than crashing.
- TileDB write path optimized (consecutive-metadata dedup, vectorized ID strings, structured-array views) to reduce memory churn at scale, with cleanup on metadata-DB failure.
- Transcription and voice-recording now use proper Qt worker threads; OCR pre-check uses threads instead of a spawn pool.
- Assorted fixes: OpenAI/LM Studio streaming error handling, documentation-scraper path matching, TTS normalization guard, vision test text wrapping, model-download index handling, and a proper
__main__guard, etc.
🧰 Dependencies & Build
- Broad dependency refresh across the stack (PyTorch CUDA wheels, transformers, sentence-transformers, PySide6, pymupdf, and many more), with carefully chosen version caps documented inline to keep the environment consistent.
- Added
tools/build_user_manual_db.pyto fully automate building, zipping, and deploying the Ask Jeeves knowledge-base database. setup_windows.pynow enables ANSI color output in the Windows console.
Contributors
octo-patch