koboldcpp-1.118

koboldcpp-1.118

• Row Split has been removed, selecting it will now default to tensor split.
• Rocwmma has been removed, matching upstream behavior.
• Added support for qwen3tts aliases for all voices.
• Breaking change for RPC to match upstream behavior, allowing llama.cpp RPC server and clients to be interchangeable with koboldcpp ones again.
• Fixed loading for SeFI image.
• Fixed a quality regression in Qwen Image and Qwen Image Edit.
• Fixed an issue with incorrect GPU selection when multi gpu is used.
• Fix for multi-turn deepseek v4 flash prompt process issues.
• Increased default server thread counts
• Added more information when running in batched (parallel requests) mode
• --jinjathink flag automatically enables jinja
• Added support for loading int8 convrot image models, exclude z-image from models that support image references.
• Image generation abort by @wbruna - Now when generating an image, you can abort the generation by closing the connection. This behavior can be disabled with the keep_image_gen_on_disconnect request field. Pressing cancel in SDUI will also do this.
• Also synced new version of sd.cpp from @wbruna
• Fixed generate more button being broken when whisper model is no longer loaded by @Anai-Guo
• Set no_host by default, please report if performance issues occur.
• Added multiple language support for qwen3tts, usable only with XTTS endpoint currently.
• Updated Kobold Lite, multiple fixes and improvements
  • Placeholder context handling improvements by @hu-yijie, this is not heavily tested so please report any issues faced.
• Merged fixes, new model support, and improvements from upstream

Download and run the koboldcpp.exe (Windows) or koboldcpp-linux-x64 (Linux), which is a one-file pyinstaller for NVIDIA GPU users.
If you have an older CPU or older NVIDIA GPU and koboldcpp does not work, try oldpc version instead (Cuda11 + AVX1).
If you don't have an NVIDIA GPU, or do not need CUDA, you can use the nocuda version which is smaller.
If you're using AMD, we recommend trying the Vulkan option in the nocuda build first, for best support. Alternatively, you can download our rolling ROCm binary here if you use Linux.
If you're on a modern MacOS (M-Series) you can use the koboldcpp-mac-arm64 MacOS binary.
Click here for .gguf conversion and quantization tools
Newer rolling experimental builds can be found here, these are auto-updated and may be unstable.

Run it from the command line with the desired launch parameters (see --help), or manually select the model in the GUI.
and then once loaded, you can connect like this (or use the full koboldai client):
http://localhost:5001

For more information, be sure to run the program from command line with the --help flag. You can also refer to the readme and the wiki.