-
Notifications
You must be signed in to change notification settings - Fork 131
jevos vs jev vs laya
For yes/no decisions on a CPU, jevos is the fastest of the three on short and long requests, TypeSafe's hosted Jev is the most accurate, and Laya is the one to pick when you need many languages or scores past jevos's early ones. On the same laptop and the same requests, jevos answered in 26 ms and 112 ms, Laya in 104 ms and 449 ms, and Jev in about 345 ms both times, most of it network. On 2,000 yes/no questions about business policies none of them had been tuned on, Jev was right 0.927 of the time, jevos 0.810, and Laya 0.489.
A conflict of interest, stated first: we build jevos. Every number below was measured on the same requests or the same questions for all three, and the places where jevos loses are in the tables, not in footnotes.
This page is what each one can answer, the speed and accuracy measurements and how they were taken, and which one fits which job.
| jevos | Jev | Laya | |
|---|---|---|---|
| Yes/no questions | yes | yes | yes |
| Multiple choice | yes | yes | yes |
| Scores | early | yes | yes |
| Runs on | your machine | TypeSafe's cloud | your machine |
| Cost | free | per token | free |
| Context | 8,192 tokens | not stated | 512 tokens (English checkpoint) |
| Languages | English | see TypeSafe's docs | 100+ |
| Runtime | OpenVINO, CPU | hosted API | PyTorch, CPU or GPU |
All three answer the same three primitives in principle: noul (yes/no, as a probability),
choice and score, and none of them generates text to get there. jevos speaks the same wire
format as Jev, so code written for Jev's SDK runs against a jevos server unchanged for yes/no
and choice questions, and for score questions, whose answers are early (54% on held-out score
questions, 82% within one level).
Laya is an encoder model (ModernBERT-large, 421M parameters, per its own README) with an English checkpoint and a multilingual one. The numbers here use the English checkpoint as shipped.
Same two requests on the same laptop, an Intel Core Ultra 7 255H with 16 threads and no GPU in use; Jev through its hosted API from Europe:
| short request | long request | |
|---|---|---|
| jevos (OpenVINO, CPU, INT8) | 26 ms | 112 ms |
| Laya, English checkpoint (PyTorch, CPU) | 104 ms | 449 ms |
| Jev (hosted API, network included) | 344 ms | 345 ms |
The short request is about 30 tokens and the long one about 190. The two local models grow with the length of the text; Jev's time barely changes, because it is dominated by the round trip, so on a very long document the hosted model's relative cost falls. From a server closer to TypeSafe's, Jev's numbers would likely be lower; from a laptop in Europe, this is what an application sees. What "fastest" can honestly mean, and why nobody can claim the fastest model in general, is on the fastest AI model for yes/no decisions.
2,000 yes/no questions on three business policies, with every answer computed by code from the rule and the facts, identical for all three:
| accuracy | |
|---|---|
| Jev | 0.927 |
| jevos | 0.810 |
| Laya (zero-shot, English checkpoint) | 0.489 |
Jev is clearly stronger on rules, and the gap is largest on additive point scores, where several signals are summed and compared with a threshold, which is arithmetic, the weakest skill of a small model (measured here).
Laya's result needs its context: it was used as shipped, with no tuning on these tasks, and its 512-token context truncates the longer rule texts, which explains most of its score. It is not a measure of Laya on the tasks it was built and documented for.
jevos if the decision is yes/no or multiple choice, the text is English, and you want it local: on a laptop, in CI, on a server without a GPU, or anywhere the text should not leave the machine. It is the fastest of the three on both request sizes, and free. Plan around its weak spot, computation, by doing arithmetic in code.
Jev if accuracy on hard rules matters more than latency and cost, if you need scores past jevos's early ones, or if you do not want to run anything. Because the wire format is the same, starting with jevos and moving the hard cases to Jev is a change of URL, not of code. The switch itself, step by step, is on an open-source alternative to Jev, and the same trade-off against a general hosted chat API is on jevos vs the OpenAI API.
Laya if you need many languages, or local scores past jevos's early ones, and your texts fit its context.
Is jevos an alternative to Jev? For yes/no and multiple-choice questions, yes: same wire format, local, free and faster from a laptop. Scores are answered too, but early. Jev is more accurate on unseen rules (0.927 against 0.810).
Which is fastest? jevos: 26 ms and 112 ms against 104/449 ms for Laya and about 345 ms for Jev on our two requests.
Which is most accurate? Jev, on our 2,000 policy questions.
Can jevos replace Laya? For English yes/no decisions with texts longer than 512 tokens, it is faster and more accurate in our test. For other languages, not yet; for scores, only where an answer within one level is good enough.
Does my data leave my machine? With jevos and Laya, no. With Jev, the text is sent to TypeSafe's API.
Are jevos and TypeSafe related? No. jevos is an independent project; "TypeSafe" and Jev belong to TypeSafe AI.
See also: ask a local LLM a yes/no question, LLM policy decisions: put the rule in the question and our held-out benchmark said 0.855, which explains why we report the 2,000-question number.
- Latency and accuracy tables, and the feature table: the jev README, measured by us on the same requests and questions for all three.
- Laya's model, languages, context and runtime: the Laya repository, fetched 2026-09-29.
- Jev's question types: TypeSafe's documentation, fetched 2026-09-29.
From the notes of jev, a yes/no decision model that runs on a laptop CPU. We would rather you picked the right one of the three than the one we build.
- Ask a local LLM a yes/no question and get P(yes)
- Zero-shot text classification with yes/no questions
- LLM policy decisions: put the rule in the question
- LLM as a judge on a CPU
- Why a small LLM says yes when the answer is no
- Small LLMs and arithmetic in yes/no questions
- Our held-out benchmark said 0.855, new questions said 0.757
- jevos vs Jev vs Laya for yes/no decisions
- An open-source alternative to Jev for yes/no decisions
- jevos vs the OpenAI API for yes/no classification
- jevos vs Ollama for yes/no decisions
- jevos vs bart-large-mnli for zero-shot classification
- A yes/no LLM vs a fine-tuned BERT classifier
- jevos vs SetFit: zero-shot vs few-shot classification
- jevos vs Llama Guard for content safety checks
- jev serve vs llama.cpp server for classification
- jevos vs LM Studio: a decision server, not a chat app
- Local vs hosted LLM decisions: latency, cost, privacy
- A yes/no LLM vs a business rules engine
- LLM decisions vs keyword rules and regex
- The fastest AI model for yes/no decisions
- What makes a local LLM fast on a CPU
- Why one forward pass beats generating an answer
- Prefill vs decode: where LLM latency comes from
- Why LLM latency grows with the length of the text
- Why a hosted LLM API cannot answer in 50 ms
- Many questions about one text: why the extra ones are cheap
- CPU or GPU for a small LLM
- Latency budgets: where a 200 ms model fits
- Measuring LLM latency: median, p90 and warm-up
- Q4_K_M vs Q8_0: speed and size for a small model
- Throughput vs latency for a decision server
- What P(yes) means, and what it does not
- LLM calibration explained with yes/no answers
- Expected calibration error (ECE), explained
- Temperature scaling for LLM probabilities
- Platt scaling for a yes/no model
- Reading a reliability diagram
- How to choose a threshold for P(yes)
- Thresholds when a wrong yes costs more than a wrong no
- Human in the loop AI with a review band
- Precision and recall at a P(yes) threshold
- Base rates: why a 0.9 yes can still be wrong often
- Combining yes/no answers with AND, OR and NOT
- Logits, log-odds and P(yes)
- LLM confidence scores: probabilities vs self-reports
- How to write yes/no questions an LLM answers well
- Negation in yes/no questions for an LLM
- One condition per question: splitting compound questions
- Ask whether the text says it at all
- Scores as yes/no thresholds: is it at least high?
- Sending JSON as the text: designing the state
- Why wording changes an LLM's answer, and how to test it
- Mainly about: questions for messages with several topics
- Yes/no questions about tone and emotion
- Asking about intent: what does the writer want?
- Yes/no questions about long documents
- Using an English-only LLM with other languages
- Content moderation with a local LLM
- A Discord moderation bot with a local LLM
- Spam detection with yes/no questions
- Review moderation with a local LLM
- Email triage with a local LLM
- Support ticket routing with yes/no questions
- Urgency detection in customer messages
- Sentiment analysis with yes/no questions
- Intent detection with a local LLM
- Lead qualification with yes/no questions
- Fraud case triage with a local LLM
- Phishing email screening with a local LLM
- Log and alert triage with a local LLM
- Checking text for personal data with yes/no questions
- Prompt injection screening with a small model
- Document classification with a local LLM
- Product categorization with yes/no questions
- Contract clause detection with a local LLM
- Refund request triage with a local LLM
- Detecting cancellation intent in customer messages
- RAG evaluation with yes/no questions
- RAG faithfulness check with a local LLM
- Hallucination detection with a local LLM
- LLM regression tests in CI with yes/no checks
- Rubric design for an LLM judge
- Pairwise comparison with a yes/no judge
- LLM judge bias and how to control it
- Evaluation metrics for yes/no classifiers
- Building a yes/no test set for your own data
- Accuracy by kind of question: why one number hides failures
- Generating test questions with answers computed by code
- Benchmark contamination and truly held-out tests
- An LLM router with yes/no questions
- A model cascade: small model first, large model on doubt
- Semantic routing vs yes/no questions
- Gating AI agent tool calls with yes/no checks
- AI agent guardrails with yes/no questions
- Stop conditions for AI agents
- Logging LLM decisions for audit
- Reducing LLM cost with local yes/no decisions
- Replacing chat LLM calls with yes/no questions
- Structured output vs a probability
- A Python client for local LLM decisions
- Calling a local LLM decision server from JavaScript
- Local LLM yes/no decisions in n8n
- A Slack bot that uses local LLM decisions
- Home Assistant automations with local LLM decisions
- A LangChain tool for local yes/no decisions
- Batch decisions from files with jev decide
- Running LLM yes/no checks in GitHub Actions
- Securing a local LLM server with an API key
- curl examples for a local LLM decision API
- Self-hosted AI for decisions
- A private LLM for text classification
- On-premise LLM for business decisions
- GDPR and automated decision-making with an LLM
- Offline AI for decisions: no network needed
- Edge AI decisions on a CPU
- Run an LLM locally without a GPU
- Small language models explained
- When a small model is enough, and when it is not
- An LLM on a laptop: what it can do in real time
- What is GGUF, for someone deploying a classifier
- GGUF quantization types explained: Q4_K_M, Q8_0 and others
- GGUF vs safetensors
- llama.cpp vs Ollama for a classification service
- llama-cpp-python vs calling llama.cpp through ctypes
- llama.cpp on Windows without compiling
- Running llama.cpp CPU only
- Using llama.cpp prebuilt binaries instead of building