-
Notifications
You must be signed in to change notification settings - Fork 132
offline ai for decisions
Once the jev folder, with its binary, OpenVINO's libraries and the model, is on the machine,
jevos needs no network to answer: jev serve and jev decide read local files and talk to
nobody. The one-time download can happen on a connected machine, the files can be carried
into an air-gapped network, and each one can be checked against a published sha256 before it
is trusted. After that the decisions keep working with the cable unplugged.
The part people forget is the verification. An offline machine cannot re-download a file that arrived corrupted or was swapped along the way, so the checksum is the only link between what you run and what was published.
This page is what needs the network and when, what runs without it, how to move everything into an isolated network, how to verify the files, and what being offline does not give you.
Two downloads, both before the first decision, both from the release page:
| What | How | Size or note |
|---|---|---|
| jev binary |
jev-linux-x64.tar.gz, jev-windows-x64.zip or jev-macos-arm64.tar.gz
|
a jev/ folder with the binary, OpenVINO's libraries and the licenses |
| Model | jevos-v2-openvino-int8.zip |
unpacked into the jev folder as jev/model
|
Both are prebuilt archives. Nothing is compiled and nothing is installed, which matters offline: there is no Python environment, package manager or compiler toolchain to carry over.
Everything that makes a decision. jev serve loads the model from the model folder beside
the binary, or from the folder you name with --model-dir, and fetches nothing. jev decide
does the same for a single request file. The server listens on 127.0.0.1:8017 by default, so the calls that follow go
over the loopback interface and never touch a network card.
The model answers from the text you send and nothing else. It has no lookup, no retrieval and no call home, so there is no feature that quietly degrades when the connection disappears. The whole mechanism is on ask a local LLM a yes/no question.
The pattern is to download the release files on a connected machine, verify them, then carry them across.
- On the connected machine, download the archive for the isolated machine's platform
(
jev-linux-x64.tar.gz,jev-windows-x64.ziporjev-macos-arm64.tar.gz). Each archive is built for one operating system and processor architecture. - Download
jevos-v2-openvino-int8.zipandSHA256SUMS.txtfrom the release into the same directory. - Verify the files there (next section), then copy them onto the transfer medium.
- On the isolated machine, verify the files again after the copy, unpack the archive, unzip
the model into the
jevfolder, then start the server. - Test the whole sequence once with the network disabled before you depend on it. A wrong archive or a missing file is easier to find in a rehearsal than in an incident.
If your security process requires building the binary yourself, the repository builds it with
python -m pip install -r requirements.txt (OpenVINO's SDK, CMake, Ninja) and then
python scripts/build.py (a C++17 compiler; on Windows from a Visual Studio developer prompt),
and python tests/check.py checks the result. The first build needs network access, because
CMake downloads llama.cpp, so build on the connected side; an isolated machine should use the
release archives.
SHA256SUMS.txt lists one line per release file: the hash, a space, and the file name with a
* in front, which is the format sha256sum writes and reads. On Linux, from the directory
holding the files:
sha256sum --check --ignore-missing SHA256SUMS.txt--check reads the hashes from the file and checks each listed file; --ignore-missing skips
release files you did not download, such as the GGUF files.
On Windows, PowerShell's Get-FileHash computes SHA256 by default:
Get-FileHash .\jevos-v2-openvino-int8.zipCompare the Hash it prints with the line for that file in SHA256SUMS.txt. The sums file
writes hashes in lower case and the Microsoft examples show upper case; the letters differ, the
hash does not.
Then check what the server actually loaded. GET /health reports the SHA-256 of each model file
and a fingerprint of them all. Writing that fingerprint into every
decision log ties each answer to a verified file, as described on
logging LLM decisions for audit.
-
Security by itself. An isolated machine still needs access control. If the server binds
to a network interface inside the enclave, set
JEV_API_KEYso every call except/healthneeds a Bearer token. - Updates. A new model or runtime has to come in the same way, verified the same way, and tested on your own labelled cases before it replaces the old one.
- A fallback. Offline there is no hosted model to escalate uncertain cases to. The uncertain middle goes to a person, or waits. If you plan an escalation path like the one on a model cascade: small model first, it needs a connected side.
- More capability. The model is the same one: English only, yes/no, multiple choice and early scores, 0.810 on 2,000 questions about unseen business policies against 0.927 for the hosted Jev. Offline changes where it runs, not what it knows.
Can an LLM run completely offline? Yes, once its files are on the machine. jevos needs the network only to download the binary and the model.
Does jevos phone home? The serve and decide commands read local files and answer on
127.0.0.1; the only downloads are the release files you fetch yourself.
How do I check the model file is genuine? Compare its sha256 with SHA256SUMS.txt from
the release, using sha256sum --check or Get-FileHash, then confirm the hashes /health
reports.
Can I copy the setup between machines? Between machines of the same operating system and architecture, yes: each release archive is built for one platform.
Does it need a GPU offline? No. It runs on the CPU only.
See also: self-hosted AI for decisions, edge AI decisions on a CPU and what is GGUF.
- Commands, endpoints and the build from source: the jev README.
The format of
SHA256SUMS.txtis read from the file on the jevos release. - Release archives and what they contain: the jevos release.
-
sha256sumoptions: sha256sum(1) on man7.org, fetched 2026-09-29. -
Get-FileHashand its SHA256 default: Microsoft Learn, fetched 2026-09-29.
From the notes of jev, whose release ships a sums file next to the model so the check above is one command.
- Ask a local LLM a yes/no question and get P(yes)
- Zero-shot text classification with yes/no questions
- LLM policy decisions: put the rule in the question
- LLM as a judge on a CPU
- Why a small LLM says yes when the answer is no
- Small LLMs and arithmetic in yes/no questions
- Our held-out benchmark said 0.855, new questions said 0.757
- jevos vs Jev vs Laya for yes/no decisions
- An open-source alternative to Jev for yes/no decisions
- jevos vs the OpenAI API for yes/no classification
- jevos vs Ollama for yes/no decisions
- jevos vs bart-large-mnli for zero-shot classification
- A yes/no LLM vs a fine-tuned BERT classifier
- jevos vs SetFit: zero-shot vs few-shot classification
- jevos vs Llama Guard for content safety checks
- jev serve vs llama.cpp server for classification
- jevos vs LM Studio: a decision server, not a chat app
- Local vs hosted LLM decisions: latency, cost, privacy
- A yes/no LLM vs a business rules engine
- LLM decisions vs keyword rules and regex
- The fastest AI model for yes/no decisions
- What makes a local LLM fast on a CPU
- Why one forward pass beats generating an answer
- Prefill vs decode: where LLM latency comes from
- Why LLM latency grows with the length of the text
- Why a hosted LLM API cannot answer in 50 ms
- Many questions about one text: why the extra ones are cheap
- CPU or GPU for a small LLM
- Latency budgets: where a 200 ms model fits
- Measuring LLM latency: median, p90 and warm-up
- Q4_K_M vs Q8_0: speed and size for a small model
- Throughput vs latency for a decision server
- What P(yes) means, and what it does not
- LLM calibration explained with yes/no answers
- Expected calibration error (ECE), explained
- Temperature scaling for LLM probabilities
- Platt scaling for a yes/no model
- Reading a reliability diagram
- How to choose a threshold for P(yes)
- Thresholds when a wrong yes costs more than a wrong no
- Human in the loop AI with a review band
- Precision and recall at a P(yes) threshold
- Base rates: why a 0.9 yes can still be wrong often
- Combining yes/no answers with AND, OR and NOT
- Logits, log-odds and P(yes)
- LLM confidence scores: probabilities vs self-reports
- How to write yes/no questions an LLM answers well
- Negation in yes/no questions for an LLM
- One condition per question: splitting compound questions
- Ask whether the text says it at all
- Scores as yes/no thresholds: is it at least high?
- Sending JSON as the text: designing the state
- Why wording changes an LLM's answer, and how to test it
- Mainly about: questions for messages with several topics
- Yes/no questions about tone and emotion
- Asking about intent: what does the writer want?
- Yes/no questions about long documents
- Using an English-only LLM with other languages
- Content moderation with a local LLM
- A Discord moderation bot with a local LLM
- Spam detection with yes/no questions
- Review moderation with a local LLM
- Email triage with a local LLM
- Support ticket routing with yes/no questions
- Urgency detection in customer messages
- Sentiment analysis with yes/no questions
- Intent detection with a local LLM
- Lead qualification with yes/no questions
- Fraud case triage with a local LLM
- Phishing email screening with a local LLM
- Log and alert triage with a local LLM
- Checking text for personal data with yes/no questions
- Prompt injection screening with a small model
- Document classification with a local LLM
- Product categorization with yes/no questions
- Contract clause detection with a local LLM
- Refund request triage with a local LLM
- Detecting cancellation intent in customer messages
- RAG evaluation with yes/no questions
- RAG faithfulness check with a local LLM
- Hallucination detection with a local LLM
- LLM regression tests in CI with yes/no checks
- Rubric design for an LLM judge
- Pairwise comparison with a yes/no judge
- LLM judge bias and how to control it
- Evaluation metrics for yes/no classifiers
- Building a yes/no test set for your own data
- Accuracy by kind of question: why one number hides failures
- Generating test questions with answers computed by code
- Benchmark contamination and truly held-out tests
- An LLM router with yes/no questions
- A model cascade: small model first, large model on doubt
- Semantic routing vs yes/no questions
- Gating AI agent tool calls with yes/no checks
- AI agent guardrails with yes/no questions
- Stop conditions for AI agents
- Logging LLM decisions for audit
- Reducing LLM cost with local yes/no decisions
- Replacing chat LLM calls with yes/no questions
- Structured output vs a probability
- A Python client for local LLM decisions
- Calling a local LLM decision server from JavaScript
- Local LLM yes/no decisions in n8n
- A Slack bot that uses local LLM decisions
- Home Assistant automations with local LLM decisions
- A LangChain tool for local yes/no decisions
- Batch decisions from files with jev decide
- Running LLM yes/no checks in GitHub Actions
- Securing a local LLM server with an API key
- curl examples for a local LLM decision API
- Self-hosted AI for decisions
- A private LLM for text classification
- On-premise LLM for business decisions
- GDPR and automated decision-making with an LLM
- Offline AI for decisions: no network needed
- Edge AI decisions on a CPU
- Run an LLM locally without a GPU
- Small language models explained
- When a small model is enough, and when it is not
- An LLM on a laptop: what it can do in real time
- What is GGUF, for someone deploying a classifier
- GGUF quantization types explained: Q4_K_M, Q8_0 and others
- GGUF vs safetensors
- llama.cpp vs Ollama for a classification service
- llama-cpp-python vs calling llama.cpp through ctypes
- llama.cpp on Windows without compiling
- Running llama.cpp CPU only
- Using llama.cpp prebuilt binaries instead of building