koboldcpp-1.62 experimental build

Use Nexesenex fork instead.

https://github.com/Nexesenex/kobold.cpp/releases
Works better than mine.

---

This particular version supports CohereForAI/c4ai-command-r-v01 model.

GGUF quants is available at Hugging Face (https://huggingface.co/models?search=c4ai-command-r-v01%20gguf).
A 24GB VRAM GPU can offload 20 layers with 8k context (Q5KM).

Built from concedo_experimental with w64devkit and Visual Studio with CUDA 12.4 for CuBLAS.

Kalomaze's SillyTavern template:
Context
Instruct

Attached SillyTavern templates below are made by myself.

How to compile on Windows with CuBLAS support

Requirements:

• Latest release of w64devkit (https://github.com/skeeto/w64devkit). Download and extract w64devkit-1.21.0.zip
• Visual Studio Community (https://visualstudio.microsoft.com/downloads/) with C++ CMake tools for Windows and Windows 11 SDK
• Visual Studio Code (https://code.visualstudio.com)
• CUDA Toolkit 12.4 (https://developer.nvidia.com/cuda-downloads)

Steps:

• Open terminal and run git clone -b concedo_experimental https://github.com/knifeayumu/koboldcpp
• Run w64devkit and locate koboldcpp source folder then run make.
• Run VSCode and locate koboldcpp source folder. Open CMakeLists.txt and hit the Build button on the bottom of the window.
• Copy koboldcpp_cublas.dll located at build\bin\Debug folder and paste it to source root folder.
• Run the script make_pyinstaller_hybrid_henk.bat at a regular terminal (or Windows Explorer).
• Run koboldcpp.exe located at dist folder.