llama_cpp_python-0.3.9-cp312-cp312-win_amd64-qwen3_cuda124.whl

llama-cpp-python Custom Build for Python 3.12 & CUDA 12.8

A community-provided, up-to-date wheel for high-performance LLM inference on Windows, now supporting Qwen3.

This release provides a custom-built .whl file for llama-cpp-python with CUDA acceleration, compiled to bring modern model support to Python 3.12 environments on Windows (x64) with NVIDIA CUDA 12.8.

It was created to address the gap left by slow or inactive official releases, especially for users who need support for recent Python versions and new GGUF models like Qwen3.

Highlights

• Python 3.12 Ready — Built natively for cp312.
• CUDA 12.4 Binary — Compatible with modern NVIDIA drivers and built with CUDA 12.8 toolchain.
• Multi-GPU Support — Verified on dual-GPU system (RTX 3090 + 4060 Ti) with full layer offloading and GPU splitting.
• Latest GGUF Compatibility — Built with llama.cpp release b5602, supporting models like Qwen3.

Build & Verification

• OS: Windows 11 (x64)
• Build Tools: Visual Studio 2022 ("Desktop development with C++" workload)
• CUDA Toolkit: 12.8
• Python: 3.12.10
• llama.dll: From llama.cpp release b5602

Test Case

The wheel was successfully used to load the Qwen3-4B-Q5_K_M.gguf model with:

• n_gpu_layers = -1

All 37 model layers were offloaded and automatically distributed across both GPUs.

Why This Exists

Many “official” repos rely on a single maintainer and fall behind on updates. This wheel was built to fill that gap.

It is based on the principle that transparency and community validation matter more than an official label. Anyone can reproduce or audit this build process.

Installation

1. Download the .whl file from this release.

2. In a Python 3.12 virtual environment, run:

   pip install llama_cpp_python-[version]-cp312-cp312-win_amd64.whl

Credits

• llama-cpp-python: https://github.com/abetlen/llama-cpp-python
• llama.cpp: https://github.com/ggml-org/llama.cpp
• Huge thanks to Georgi Gerganov (@ggerganov) — original author of llama.cpp

Maintained by

Bernard Peter Fitzgerald