-
Notifications
You must be signed in to change notification settings - Fork 132
open source alternative to jev
jevos is an open-source (MIT) local server that speaks TypeSafe Jev's wire format, so yes/no
questions written for Jev move over by pointing the client at http://127.0.0.1:8017 instead of
TypeSafe's API. The request body, the jev-latest model name and the shape of the answers stay
the same, for choice and score questions too. Two things do not: score answers are early
(54% on held-out score questions, 82% within one level), and on a yes/no question Jev's optional
criteria field is optional and checked but not read, so the rule has to live in instructions
(choice and score questions require criteria, and use it). Whether the switch is worth it depends on how a local 1B model does on your questions, not on the
code.
Conflict of interest, in one line: we build jevos, and we are not affiliated with TypeSafe AI, which builds Jev.
The code change is the easy part of a migration. The part that deserves a day of work is checking accuracy on your own cases, because on 2,000 rule questions neither model had been tuned on, Jev was right 0.927 of the time and jevos 0.810. For some applications that gap is irrelevant; for others it is the whole decision.
This page is what stays the same, the switch step by step, what the local server refuses or ignores, how to confirm which model answered, and when to keep Jev.
The endpoint is POST /v1/systemone on both. A request has model, state (a string, or any
JSON object or array) and named questions; each yes/no question is
{"type": "noul", "instructions": "...?"}, and each answer comes back as
{"type": "noul", "noul": <P(yes)>} under the same name. A choice question,
{"type": "choice", "instructions": "...", "criteria": {"<option>": "<optional description>", ...}},
comes back as {"type": "choice", "choice": <most probable option>, "probabilities": {...}, "confidence": ...},
Jev's shape. The usage block is there too, with
output_tokens always 0, because jevos generates no text.
The model name also carries over. jev-latest is accepted, and so is any other jev-* name, so a
client pinned to a Jev version string does not need editing. The response always names the model
that actually answered, jevos-v2, which is how you tell the two apart in logs.
Authentication works the same way when you want it. Start the server with the JEV_API_KEY
environment variable set and every call except /health requires Authorization: Bearer <key>,
with a 401 otherwise, which is the header TypeSafe's API reference documents for Jev.
- Download the archive for your machine (
jev-linux-x64.tar.gz,jev-windows-x64.ziporjev-macos-arm64.tar.gz),jevos-v2-openvino-int8.zipandSHA256SUMS.txtfrom the release page and check the hashes. - Unpack the binary and the model, and start the server:
tar -xzf jev-linux-x64.tar.gz
cd jev
unzip ../jevos-v2-openvino-int8.zip # creates model/
./jev serve --threads 16- Change the base URL in your client. With raw HTTP, replace
https://api.typesafe.ai/v1/systemonewithhttp://127.0.0.1:8017/v1/systemone. TypeSafe's Python SDK documents abase_urlparameter and aTYPESAFE_BASE_URLenvironment variable for this, and reads the key fromTYPESAFE_API_KEY:
export TYPESAFE_BASE_URL=http://127.0.0.1:8017If the local server runs without JEV_API_KEY, it does not check the header, so whatever
placeholder key the SDK requires is enough.
- Wait for
GET /healthto return{"status": "ready", ...}before sending traffic; the model loads once and then stays resident, using about 1 GB of memory.
The same steps are covered from the other direction, starting from nothing, on ask a local LLM a yes/no question.
| Jev feature | On jevos |
|---|---|
noul questions |
answered |
choice questions |
answered, 2 to 26 options |
score questions |
answered, 2 to 10 levels (early) |
criteria on a noul
|
optional; checked, not read |
criteria on a choice or score
|
required |
| unknown fields | rejected with 422
|
| languages other than English | not supported |
The criteria line is the one that bites quietly. A request that relied on
criteria.true and criteria.false to define what yes means will still be accepted and still
get an answer, but the definition will not have been used. Move it into the question: "Our
policy refunds items reported missing within 30 days of delivery. Should this customer get a
refund?" is the README's own example of a rule written into instructions, and the reasoning
behind it is on LLM policy decisions: put the rule in the question.
A choice question is answered the way the model saw choices in training: one yes/no question
per option, each listing all the options, and each option's P(yes) divided by the sum over the
options. probabilities sum to 1, in the options' order, and confidence is Jev's: the peak
probability rescaled from uniform (0) to certain (1). The text is read once, and each option
costs about as much as one more yes/no question. When a text can fit several options at once,
separate yes/no questions per option, as in
zero-shot classification with yes/no questions,
give each option its own probability instead of a share of 1. A score becomes one "is it at least level n?"
question per boundary.
Three checks, all cheap:
-
The
modelfield of every response names the served model, not the alias you sent. -
GET /v1/modelslists the served model and thejev-latestalias; the alias entry says it is answered by the local model and not by TypeSafe's Jev. -
GET /healthreports the SHA-256 of the model's files, so a log line can prove which files made a decision.
Every successful response also carries a Server-Timing header with the inference time, which makes a
before-and-after latency comparison possible without a separate benchmark harness.
Keep Jev, or keep it for part of the traffic, when:
- Accuracy on hard rules matters most. 0.927 against 0.810 on our 2,000 policy questions, with the gap largest on additive point scores, where several signals are summed and compared with a cut-off.
- You need
scoreanswers past jevos's early ones. - Your texts are not in English.
- You do not want to operate anything. A local server is a folder and a port, and you own its uptime.
Because the wire format is shared, the split does not have to be all or nothing. The pattern on a model cascade: small model first, large model on doubt answers confident cases locally and sends the middle band to the hosted model, and the only difference between the two calls is the URL.
Is there an open-source alternative to Jev? For yes/no and multiple-choice questions, and early scores, jevos: MIT code, a model file on GitHub, and the same wire format, running on a CPU.
Do I have to change my code? For noul, choice and score questions, only the base URL.
Code that relies on criteria on a noul needs the change in the table above.
Why did my criteria stop working? On a yes/no question, jevos checks the field and does
not read it. Put the
definition of yes into instructions.
Is it faster? From a laptop in Europe, on our two requests: 26 and 112 ms locally against 344 and 345 ms for the hosted API, network included.
Is it as accurate? No. On our 2,000 policy questions Jev was right 0.927 of the time and jevos 0.810. Measure on your own cases before moving everything.
See also: jevos vs Jev vs Laya for yes/no decisions, securing a local LLM server with an API key and local vs hosted LLM decisions.
- Wire format compatibility, the
jev-*aliases, thechoiceandscoreanswers, thecriteriabehaviour,/health,/v1/models,Server-TimingandJEV_API_KEY: the jev README and the jev source. - Latency (26/112 ms against 344/345 ms) and accuracy (0.927 against 0.810): our own measurements, published in the README.
- Jev's base URL, the Bearer header and the SDK's
base_url,TYPESAFE_BASE_URLandTYPESAFE_API_KEY: TypeSafe API reference and Python SDK usage, fetched 2026-09-29.
From the notes of jev, which kept the wire format so that leaving it, in either direction, is a one-line change.
- Ask a local LLM a yes/no question and get P(yes)
- Zero-shot text classification with yes/no questions
- LLM policy decisions: put the rule in the question
- LLM as a judge on a CPU
- Why a small LLM says yes when the answer is no
- Small LLMs and arithmetic in yes/no questions
- Our held-out benchmark said 0.855, new questions said 0.757
- jevos vs Jev vs Laya for yes/no decisions
- An open-source alternative to Jev for yes/no decisions
- jevos vs the OpenAI API for yes/no classification
- jevos vs Ollama for yes/no decisions
- jevos vs bart-large-mnli for zero-shot classification
- A yes/no LLM vs a fine-tuned BERT classifier
- jevos vs SetFit: zero-shot vs few-shot classification
- jevos vs Llama Guard for content safety checks
- jev serve vs llama.cpp server for classification
- jevos vs LM Studio: a decision server, not a chat app
- Local vs hosted LLM decisions: latency, cost, privacy
- A yes/no LLM vs a business rules engine
- LLM decisions vs keyword rules and regex
- The fastest AI model for yes/no decisions
- What makes a local LLM fast on a CPU
- Why one forward pass beats generating an answer
- Prefill vs decode: where LLM latency comes from
- Why LLM latency grows with the length of the text
- Why a hosted LLM API cannot answer in 50 ms
- Many questions about one text: why the extra ones are cheap
- CPU or GPU for a small LLM
- Latency budgets: where a 200 ms model fits
- Measuring LLM latency: median, p90 and warm-up
- Q4_K_M vs Q8_0: speed and size for a small model
- Throughput vs latency for a decision server
- What P(yes) means, and what it does not
- LLM calibration explained with yes/no answers
- Expected calibration error (ECE), explained
- Temperature scaling for LLM probabilities
- Platt scaling for a yes/no model
- Reading a reliability diagram
- How to choose a threshold for P(yes)
- Thresholds when a wrong yes costs more than a wrong no
- Human in the loop AI with a review band
- Precision and recall at a P(yes) threshold
- Base rates: why a 0.9 yes can still be wrong often
- Combining yes/no answers with AND, OR and NOT
- Logits, log-odds and P(yes)
- LLM confidence scores: probabilities vs self-reports
- How to write yes/no questions an LLM answers well
- Negation in yes/no questions for an LLM
- One condition per question: splitting compound questions
- Ask whether the text says it at all
- Scores as yes/no thresholds: is it at least high?
- Sending JSON as the text: designing the state
- Why wording changes an LLM's answer, and how to test it
- Mainly about: questions for messages with several topics
- Yes/no questions about tone and emotion
- Asking about intent: what does the writer want?
- Yes/no questions about long documents
- Using an English-only LLM with other languages
- Content moderation with a local LLM
- A Discord moderation bot with a local LLM
- Spam detection with yes/no questions
- Review moderation with a local LLM
- Email triage with a local LLM
- Support ticket routing with yes/no questions
- Urgency detection in customer messages
- Sentiment analysis with yes/no questions
- Intent detection with a local LLM
- Lead qualification with yes/no questions
- Fraud case triage with a local LLM
- Phishing email screening with a local LLM
- Log and alert triage with a local LLM
- Checking text for personal data with yes/no questions
- Prompt injection screening with a small model
- Document classification with a local LLM
- Product categorization with yes/no questions
- Contract clause detection with a local LLM
- Refund request triage with a local LLM
- Detecting cancellation intent in customer messages
- RAG evaluation with yes/no questions
- RAG faithfulness check with a local LLM
- Hallucination detection with a local LLM
- LLM regression tests in CI with yes/no checks
- Rubric design for an LLM judge
- Pairwise comparison with a yes/no judge
- LLM judge bias and how to control it
- Evaluation metrics for yes/no classifiers
- Building a yes/no test set for your own data
- Accuracy by kind of question: why one number hides failures
- Generating test questions with answers computed by code
- Benchmark contamination and truly held-out tests
- An LLM router with yes/no questions
- A model cascade: small model first, large model on doubt
- Semantic routing vs yes/no questions
- Gating AI agent tool calls with yes/no checks
- AI agent guardrails with yes/no questions
- Stop conditions for AI agents
- Logging LLM decisions for audit
- Reducing LLM cost with local yes/no decisions
- Replacing chat LLM calls with yes/no questions
- Structured output vs a probability
- A Python client for local LLM decisions
- Calling a local LLM decision server from JavaScript
- Local LLM yes/no decisions in n8n
- A Slack bot that uses local LLM decisions
- Home Assistant automations with local LLM decisions
- A LangChain tool for local yes/no decisions
- Batch decisions from files with jev decide
- Running LLM yes/no checks in GitHub Actions
- Securing a local LLM server with an API key
- curl examples for a local LLM decision API
- Self-hosted AI for decisions
- A private LLM for text classification
- On-premise LLM for business decisions
- GDPR and automated decision-making with an LLM
- Offline AI for decisions: no network needed
- Edge AI decisions on a CPU
- Run an LLM locally without a GPU
- Small language models explained
- When a small model is enough, and when it is not
- An LLM on a laptop: what it can do in real time
- What is GGUF, for someone deploying a classifier
- GGUF quantization types explained: Q4_K_M, Q8_0 and others
- GGUF vs safetensors
- llama.cpp vs Ollama for a classification service
- llama-cpp-python vs calling llama.cpp through ctypes
- llama.cpp on Windows without compiling
- Running llama.cpp CPU only
- Using llama.cpp prebuilt binaries instead of building