Repository navigation
This update adds support for running very large mixture-of-experts models on a normal PC, using Colibri by JustVugg.
What's new
- When you download GLM-5.2, Kimi K3, Inkling, or OLMoE (SafeTensors), Sytra can serve them through Colibri. The first time you do this, Sytra installs Colibri for you. You can also grab the engine from Colibri's releases.
- Regular GGUF models still use llama.cpp. Kimi K2.7 Code still uses Sytra's own engine.
- Large models on a 12 GB GPU will be slow. Sytra tells you that instead of promising 5 tokens per second.
Update
npx -y --prefer-online github:walkowicz19/sytra-studio#1.2.0-fix mcp
If you already have Sytra 1.2.0 installed, this refresh copies the new scripts automatically.
Credits
Disk-streamed serving for those models uses Colibri.