koboldcpp-1.120
- Minor fix to assistant gen prefills being triggered incorrectly
- Fixed a bug where failsafe mode was incorrectly selected
- Added support for DirectIO model load mode (
--usedirectio), also mlock and mmap can now be combined. - New model support: Qwen3.8-Flash-Next and Ling-3.0-flash are fully supported. Note that there are bad quants floating out there, so make sure you use the right files.
- Image generation sync and update from @wbruna, various image generation fixes
- Updated Kobold Lite, multiple fixes and improvements
- Added custom user-configurable javascript tools by @hu-yijie, compatible with standard tool calling.
- Merged fixes, new model support, and improvements from upstream
Download and run the koboldcpp.exe (Windows) or koboldcpp-linux-x64 (Linux), which is a one-file pyinstaller for NVIDIA GPU users.
If you have an older CPU or older NVIDIA GPU and koboldcpp does not work, try oldpc version instead (Cuda11 + AVX1).
If you don't have an NVIDIA GPU, or do not need CUDA, you can use the nocuda version which is smaller.
If you're using AMD, we recommend trying the Vulkan option in the nocuda build first, for best support. Alternatively, you can download our rolling ROCm binary here if you use Linux.
If you're on a modern MacOS (M-Series) you can use the koboldcpp-mac-arm64 MacOS binary.
Click here for .gguf conversion and quantization tools
Newer rolling experimental builds can be found here, these are auto-updated and may be unstable.
Run it from the command line with the desired launch parameters (see --help), or manually select the model in the GUI.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001
For more information, be sure to run the program from command line with the --help flag. You can also refer to the readme and the wiki.
