-
Notifications
You must be signed in to change notification settings - Fork 132
llama cpp on windows
You do not need Visual Studio to run llama.cpp on Windows: every llama.cpp release publishes
ready-made Windows zips for the CPU (x64 and arm64) and for GPU backends such as CUDA and
Vulkan; unzip one and its tools are ready. --list-devices then shows which compute devices
the build can see, and --device none runs the model on the processor whatever else is
installed. jev needs none of this: jev-windows-x64.zip holds one native binary that runs the
model on the CPU, and the jevos-v2 release ships the model as GGUF files for llama.cpp.
Building llama.cpp yourself on Windows is supported but heavy: the build guide asks for Visual Studio 2022 with the C++ desktop workload, CMake tools, Git, the Clang compiler and the LLVM toolset for MSBuild, and GPU builds add the CUDA toolkit or the Vulkan SDK. For running a model, none of that is necessary.
This page is the ways to get llama.cpp without compiling, which Windows archive does what, how to check the result, how jev runs on Windows, and the usual snags.
-
The releases page. Each build of llama.cpp is published as a tag (
bplus a number) on GitHub with archives per platform and backend. Unzip one and the tools and libraries are ready. -
winget. The Hugging Face llama.cpp guide gives
winget install llama.cppfor Windows. - A tool that bundles it. LM Studio and Ollama run models with llama.cpp and install it for you, behind their own programs and APIs.
Options 1 and 2 give you llama.cpp's own programs, which is what the rest of this page is about; loading its library from Python instead is compared on llama-cpp-python vs calling llama.cpp through ctypes.
For release b11081, for example, the Windows packages include:
| Machine | Family | Archive |
|---|---|---|
| x64 | CPU | llama-b11081-bin-win-cpu-x64.zip |
| arm64 | CPU | llama-b11081-bin-win-cpu-arm64.zip |
| x64 | Vulkan (AMD, Intel and NVIDIA GPUs) | llama-b11081-bin-win-vulkan-x64.zip |
| x64 | CUDA 13.4 |
llama-b11081-bin-win-cuda-13.4-x64.zip plus cudart-llama-bin-win-cuda-13.4-x64.zip
|
| x64 | ROCm 10.0 | llama-b11081-bin-win-rocm-10.0-x64.zip |
| x64 | SYCL | llama-b11081-bin-win-sycl-x64.zip |
The release itself has more (a CUDA 12.4 build, for example). The CUDA build comes as two zips because the CUDA runtime libraries are shipped separately; both are unpacked into one folder.
llama.cpp's tools take --list-devices, which prints the compute devices the unpacked build can
see. Look for the device you expect, with the kind you expect, before measuring anything.
On llama.cpp's tools, --device decides where the model runs, separately from which archive was
unpacked: --device none (or -ngl 0) keeps every weight on the processor, even in a GPU build.
jev runs on the CPU only (x86-64 with AVX2), with no GPU path, no Python and no download step.
Unzip jev-windows-x64.zip, unpack jevos-v2-openvino-int8.zip into the jev folder so that
it creates jev\model, and start the server from that folder. The jevos numbers were measured
on the CPU with no GPU; in PowerShell the command is:
.\jev.exe serveOn a laptop, running llama.cpp CPU only covers the thread setting, and what an LLM on a laptop can do in real time covers what to expect.
-
DLLs next to each other. The release zips put
llama.dll, the ggml libraries and the backend plug-ins in one folder, and the CUDA and OpenMP dependencies beside them. If you move files around by hand, keep them together. The same goes for thejevfolder, where the binary sits beside OpenVINO's libraries. -
Your own build. jev builds from source with
python -m pip install -r requirements.txtandpython scripts/build.py(CMake, Ninja, a C++17 compiler), run on Windows from a Visual Studio developer prompt. -
The wrong family picked. If
--list-devicesshows a GPU you did not want to use, you do not need to reinstall: pass--device none.
Being straight about the limit: we have not published Windows-specific timings. The reference numbers (26 ms for a short request, 112 ms for a long one) are from one laptop with an Intel Core Ultra 7 255H and 16 threads, and apply to that machine.
Can I run llama.cpp on Windows without building it? Yes. Each release has Windows zips for CPU and GPU backends, and winget can install it too.
Which llama.cpp zip do I need on Windows? For CPU only, win-cpu-x64 (or win-cpu-arm64
on ARM). For an NVIDIA GPU, the CUDA zip plus its cudart zip. For other GPUs, Vulkan.
Does jev need llama.cpp on Windows? No. jev-windows-x64.zip holds the binary and
OpenVINO's libraries; llama.cpp's tokenizer is compiled in.
How do I see which devices llama.cpp detects? llama.cpp's own tools have a
--list-devices option.
Does jevos need a GPU on Windows? No. jev runs on the CPU only.
See also: using llama.cpp prebuilt binaries, run an LLM locally without a GPU and CPU or GPU for a small LLM.
- Our own facts: the contents of
jev-windows-x64.zip, how to run it and how to build it, from the jev repository; latency on the reference laptop, from the README. -
llama.cpp build guide,
fetched 2026-09-29: Windows build requirements, multiple backends,
--list-devices. - llama.cpp release b11081 and the releases page, fetched 2026-09-29.
-
Hugging Face docs: GGUF usage with llama.cpp,
fetched 2026-09-29:
winget install llama.cpp.
From the notes of jev, which ships for Windows as one folder: the binary, OpenVINO's libraries and the licenses.
- Ask a local LLM a yes/no question and get P(yes)
- Zero-shot text classification with yes/no questions
- LLM policy decisions: put the rule in the question
- LLM as a judge on a CPU
- Why a small LLM says yes when the answer is no
- Small LLMs and arithmetic in yes/no questions
- Our held-out benchmark said 0.855, new questions said 0.757
- jevos vs Jev vs Laya for yes/no decisions
- An open-source alternative to Jev for yes/no decisions
- jevos vs the OpenAI API for yes/no classification
- jevos vs Ollama for yes/no decisions
- jevos vs bart-large-mnli for zero-shot classification
- A yes/no LLM vs a fine-tuned BERT classifier
- jevos vs SetFit: zero-shot vs few-shot classification
- jevos vs Llama Guard for content safety checks
- jev serve vs llama.cpp server for classification
- jevos vs LM Studio: a decision server, not a chat app
- Local vs hosted LLM decisions: latency, cost, privacy
- A yes/no LLM vs a business rules engine
- LLM decisions vs keyword rules and regex
- The fastest AI model for yes/no decisions
- What makes a local LLM fast on a CPU
- Why one forward pass beats generating an answer
- Prefill vs decode: where LLM latency comes from
- Why LLM latency grows with the length of the text
- Why a hosted LLM API cannot answer in 50 ms
- Many questions about one text: why the extra ones are cheap
- CPU or GPU for a small LLM
- Latency budgets: where a 200 ms model fits
- Measuring LLM latency: median, p90 and warm-up
- Q4_K_M vs Q8_0: speed and size for a small model
- Throughput vs latency for a decision server
- What P(yes) means, and what it does not
- LLM calibration explained with yes/no answers
- Expected calibration error (ECE), explained
- Temperature scaling for LLM probabilities
- Platt scaling for a yes/no model
- Reading a reliability diagram
- How to choose a threshold for P(yes)
- Thresholds when a wrong yes costs more than a wrong no
- Human in the loop AI with a review band
- Precision and recall at a P(yes) threshold
- Base rates: why a 0.9 yes can still be wrong often
- Combining yes/no answers with AND, OR and NOT
- Logits, log-odds and P(yes)
- LLM confidence scores: probabilities vs self-reports
- How to write yes/no questions an LLM answers well
- Negation in yes/no questions for an LLM
- One condition per question: splitting compound questions
- Ask whether the text says it at all
- Scores as yes/no thresholds: is it at least high?
- Sending JSON as the text: designing the state
- Why wording changes an LLM's answer, and how to test it
- Mainly about: questions for messages with several topics
- Yes/no questions about tone and emotion
- Asking about intent: what does the writer want?
- Yes/no questions about long documents
- Using an English-only LLM with other languages
- Content moderation with a local LLM
- A Discord moderation bot with a local LLM
- Spam detection with yes/no questions
- Review moderation with a local LLM
- Email triage with a local LLM
- Support ticket routing with yes/no questions
- Urgency detection in customer messages
- Sentiment analysis with yes/no questions
- Intent detection with a local LLM
- Lead qualification with yes/no questions
- Fraud case triage with a local LLM
- Phishing email screening with a local LLM
- Log and alert triage with a local LLM
- Checking text for personal data with yes/no questions
- Prompt injection screening with a small model
- Document classification with a local LLM
- Product categorization with yes/no questions
- Contract clause detection with a local LLM
- Refund request triage with a local LLM
- Detecting cancellation intent in customer messages
- RAG evaluation with yes/no questions
- RAG faithfulness check with a local LLM
- Hallucination detection with a local LLM
- LLM regression tests in CI with yes/no checks
- Rubric design for an LLM judge
- Pairwise comparison with a yes/no judge
- LLM judge bias and how to control it
- Evaluation metrics for yes/no classifiers
- Building a yes/no test set for your own data
- Accuracy by kind of question: why one number hides failures
- Generating test questions with answers computed by code
- Benchmark contamination and truly held-out tests
- An LLM router with yes/no questions
- A model cascade: small model first, large model on doubt
- Semantic routing vs yes/no questions
- Gating AI agent tool calls with yes/no checks
- AI agent guardrails with yes/no questions
- Stop conditions for AI agents
- Logging LLM decisions for audit
- Reducing LLM cost with local yes/no decisions
- Replacing chat LLM calls with yes/no questions
- Structured output vs a probability
- A Python client for local LLM decisions
- Calling a local LLM decision server from JavaScript
- Local LLM yes/no decisions in n8n
- A Slack bot that uses local LLM decisions
- Home Assistant automations with local LLM decisions
- A LangChain tool for local yes/no decisions
- Batch decisions from files with jev decide
- Running LLM yes/no checks in GitHub Actions
- Securing a local LLM server with an API key
- curl examples for a local LLM decision API
- Self-hosted AI for decisions
- A private LLM for text classification
- On-premise LLM for business decisions
- GDPR and automated decision-making with an LLM
- Offline AI for decisions: no network needed
- Edge AI decisions on a CPU
- Run an LLM locally without a GPU
- Small language models explained
- When a small model is enough, and when it is not
- An LLM on a laptop: what it can do in real time
- What is GGUF, for someone deploying a classifier
- GGUF quantization types explained: Q4_K_M, Q8_0 and others
- GGUF vs safetensors
- llama.cpp vs Ollama for a classification service
- llama-cpp-python vs calling llama.cpp through ctypes
- llama.cpp on Windows without compiling
- Running llama.cpp CPU only
- Using llama.cpp prebuilt binaries instead of building