What's New
- CUDA wheels now distributed via GitHub Releases — CUDA wheels exceed PyPI's 100MB limit, so they are uploaded as release assets instead of PyPI
- Strip debug symbols from CUDA wheels to reduce size
- Unit tests + CI — added pure Python unit tests and GitHub Actions test workflow (Python 3.10–3.13)
- Bug fixes from v0.1.2: broadened VNNI check, added KV cache clear before generate
Install
CPU (PyPI)
pip install ik-llama-cpp-python==0.1.3
CUDA (GitHub Release)
# Replace cp312 with your Python version (cp310, cp311, cp312, cp313)
pip install https://github.com/gongpx20069/ik-llama-cpp-python/releases/download/v0.1.3/ik_llama_cpp_python_cuda-0.1.3-cp312-cp312-manylinux_2_28_x86_64.whl
Supported Platforms
| Platform |
Wheels |
| Linux x86_64 |
CPU (AVX2), CUDA 12.4 |
| Linux aarch64 |
CPU |
| macOS arm64 |
CPU + Metal |
| Windows x86_64 |
CPU (AVX2) |
