Repository navigation
what is a decision model
A decision model is a model that reads a text and a question with a fixed set of allowed answers, and returns a probability for each allowed answer in one pass instead of writing a reply. The answers are typed: a yes/no question gets P(yes), a multiple-choice question gets a probability per option, and a score question gets a probability per level of a scale. Your code then acts on the numbers, usually by comparing them with a threshold. TypeSafe's Jev, Microsoft-Decision-1 and jevos are three examples: the first two are hosted, jevos runs on your own CPU.
The non-obvious part is where the decision ends up. A chat model hands back a sentence to parse and trust; a decision model hands back a number, and the rule that turns it into an action lives in your code, where it can be tested, logged and changed without touching the model. Microsoft's developer post says these models make "quick, focused decisions" and "are sometimes described as" System One models.
This page is what goes in and comes out, three models that work this way, why the output is a probability, and how a decision model differs from a chat LLM, a classifier trained for one task and a rules engine.
A state goes in (the text to decide about, a string or any JSON), with named questions, each
with a type and its allowed answers. One answer per question comes back under the same name, and
no text is generated (jevos reports output_tokens 0). The multiple-choice example from the
jevos README:
{"model": "jev-latest",
"state": {"item": "wireless mouse", "customer_message": "The box arrived empty. This is the second time!"},
"questions": {"team": {"type": "choice",
"instructions": "Which team should handle this message?",
"criteria": {"billing": "payments, invoices, refunds",
"shipping": "deliveries, missing or damaged parcels",
"tech": "a product that does not work"}}}}jevos-v4 answered shipping, with probabilities 0.02, 0.93 and 0.05 and a confidence of 0.89.
The three question types, in this wire format's names:
| type | asks | gets back |
|---|---|---|
noul |
is this true? | P(yes), from 0 to 1 |
choice |
which of these options? | the most probable option, a probability per option, a confidence |
score |
how much, on this scale? | the expected level, a probability per level, a confidence |
The allowed answers are the point. A decision model cannot reply "it depends" or invent a fourth team; it can only spread probability over the answers you allowed.
- Jev, by TypeSafe AI, is hosted. jevos speaks its wire format, so code written for Jev's SDK works unchanged against jevos for yes/no questions.
- Microsoft-Decision-1, announced on 2026-10-09, runs in Microsoft Foundry and through OpenRouter. Microsoft puts the category this way: "Unlike LLMs, which are designed to generate text or reason through complex problems, decision models are purpose-built to deliver structured outputs that software can immediately act on." Its documentation uses the same three question types.
- jevos is open source (MIT): one native binary, CPU only, about 1 GB of memory, English text, 8,192 tokens of context.
A shared interface does not mean equal answers: on the six tasks in our README, Jev scored higher than jevos-v4 on all six. What a hosted option means next to a local one is on an open-source alternative to Microsoft-Decision-1.
Because the right cut-off depends on what a mistake costs, and only the application knows that. One model can serve a cautious gate and a permissive one: for example, a refund bot can require P(yes) above 0.9 to act alone, while a spam flag goes up at 0.5. A middle band can go to a person (human in the loop AI with a review band), and how to choose a threshold for P(yes) picks the numbers from labelled examples. Microsoft's documentation agrees: "Choose a probability threshold based on your workload, and validate it against labeled examples before you automate a gate."
Choice and score answers add a confidence. In the jevos README's score example, "How angry is
the customer?" on four levels (calm, annoyed, angry, furious) gets a score of 1.25 and a
confidence of 0.0: the model cannot place the message, and the README says to route such answers
to a person. A confidence is worth something only if it tracks being right: with jevos-v2, at
confidence 0.75 or more, 95% of choice answers were right, on 46% of the questions; below 0.25,
47%. More on AI decision making with probabilities and thresholds.
A chat LLM writes; a decision model scores. Generating means producing tokens one after another, then parsing what came out, which may be a label, a hedge or a paragraph. A decision model reads the text and the questions and returns numbers, so the time goes into reading the input (why one forward pass beats generating an answer) and there is nothing to parse. Asking a chat model for a JSON label fixes the shape but still gives a word instead of a probability (structured output vs a probability).
Being straight about the limit: a decision model does not explain itself, summarize or write. Microsoft's documentation says to use a generative model "when the application needs to create, summarize, rewrite, or explain content". The two combine: the decision model routes or gates, the generative one writes.
A classic text classifier learns a fixed set of labels from labelled examples of one task; new
labels mean new examples and a new model. A decision model takes the question and options at
request time, so a new label is a new line in criteria, the approach on
zero-shot text classification with yes/no questions.
The trade reverses when a task has thousands of labelled examples and never changes: a classifier
trained on exactly that data can be the better tool, as argued on
a yes/no LLM vs a fine-tuned BERT classifier.
A rules engine executes logic you wrote: if the amount is over 500 and the account is new, hold the payment. It is exact, cheap and auditable, and it cannot read "The box arrived empty. This is the second time!" and tell you the customer is upset. A decision model reads language and is never exact. They work best together: code computes what can be computed, the model is asked only what needs reading. The jevos README is explicit: on points that add up (a fraud score from six rules, measured with jevos-v2) the model was right 34% of the time, barely above the most common level, and confidently wrong, so compute sums in code and ask the parts as yes/no questions. A yes/no LLM vs a business rules engine and decision engine vs decision model go further.
What is a decision model? A model that returns a probability for each allowed answer to a typed question about a text, in one pass, without generating text. Your code applies the threshold and takes the action.
What is the difference between a decision model and an LLM? A chat LLM generates text that has to be parsed; a decision model scores a fixed set of answers. Use the first to write, the second to gate, route and grade.
What are examples of decision models? TypeSafe's Jev and Microsoft-Decision-1, both hosted, and jevos, which is open source and runs on a CPU.
Can a decision model run locally? jevos does: one binary on a CPU, about 1 GB of memory, listening on 127.0.0.1:8017. The hosted ones are reached over the network.
See also: ask a local LLM a yes/no question, what P(yes) means, and what it does not and an LLM router with yes/no questions.
- jevos facts, examples and figures, including the jevos-v2 ones and the six-task comparison with Jev: the jev README, our own measurements.
- Microsoft's definition of decision models and the announcement date: Introducing Microsoft-Decision-1, Command Line blog, dated 2026-10-09, fetched 2026-10-12.
- Question types and the advice quoted: Deploy and use Microsoft-Decision-1 in Microsoft Foundry, Microsoft Learn, fetched 2026-10-12.
- "Quick, focused decisions" and "System One": Not Every Decision Needs an LLM, Microsoft Tech Community, fetched 2026-10-12.
From the notes of jev, a decision model that answers with probabilities and leaves the deciding to your code.
- Ask a local LLM a yes/no question and get P(yes)
- Zero-shot text classification with yes/no questions
- LLM policy decisions: put the rule in the question
- LLM as a judge on a CPU
- What is a decision model in AI?
- AI decision making with probabilities and thresholds
- Why a small LLM says yes when the answer is no
- Small LLMs and arithmetic in yes/no questions
- Our held-out benchmark said 0.855, new questions said 0.757
- Does an LLM know when a fact is missing?
- Snake AI: a local model vs a cloud API
- jevos vs Jev vs Laya for yes/no decisions
- An open-source alternative to Jev for yes/no decisions
- jevos vs the OpenAI API for yes/no classification
- jevos vs Ollama for yes/no decisions
- jevos vs bart-large-mnli for zero-shot classification
- A yes/no LLM vs a fine-tuned BERT classifier
- jevos vs SetFit: zero-shot vs few-shot classification
- jevos vs Llama Guard for content safety checks
- jev serve vs llama.cpp server for classification
- jevos vs LM Studio: a decision server, not a chat app
- Local vs hosted LLM decisions: latency, cost, privacy
- A yes/no LLM vs a business rules engine
- LLM decisions vs keyword rules and regex
- An open-source alternative to Microsoft-Decision-1
- Decision engine vs decision model
- The fastest AI model for yes/no decisions
- What makes a local LLM fast on a CPU
- Why one forward pass beats generating an answer
- Prefill vs decode: where LLM latency comes from
- Why LLM latency grows with the length of the text
- Why a hosted LLM API cannot answer in 50 ms
- Many questions about one text: why the extra ones are cheap
- CPU or GPU for a small LLM
- Latency budgets: where a 200 ms model fits
- Measuring LLM latency: median, p90 and warm-up
- Q4_K_M vs Q8_0: speed and size for a small model
- Throughput vs latency for a decision server
- OpenVINO for LLM inference on a CPU
- INT8 weight quantization explained
- What P(yes) means, and what it does not
- LLM calibration explained with yes/no answers
- Expected calibration error (ECE), explained
- Temperature scaling for LLM probabilities
- Platt scaling for a yes/no model
- Reading a reliability diagram
- How to choose a threshold for P(yes)
- Thresholds when a wrong yes costs more than a wrong no
- Human in the loop AI with a review band
- Precision and recall at a P(yes) threshold
- Base rates: why a 0.9 yes can still be wrong often
- Combining yes/no answers with AND, OR and NOT
- Logits, log-odds and P(yes)
- LLM confidence scores: probabilities vs self-reports
- How to write yes/no questions an LLM answers well
- Negation in yes/no questions for an LLM
- One condition per question: splitting compound questions
- Ask whether the text says it at all
- Scores as yes/no thresholds: is it at least high?
- Sending JSON as the text: designing the state
- Why wording changes an LLM's answer, and how to test it
- Mainly about: questions for messages with several topics
- Yes/no questions about tone and emotion
- Asking about intent: what does the writer want?
- Yes/no questions about long documents
- Using an English-only LLM with other languages
- Multi-class text classification with a choice question
- Content moderation with a local LLM
- A Discord moderation bot with a local LLM
- Spam detection with yes/no questions
- Review moderation with a local LLM
- Email triage with a local LLM
- Support ticket routing with yes/no questions
- Urgency detection in customer messages
- Sentiment analysis with yes/no questions
- Intent detection with a local LLM
- Lead qualification with yes/no questions
- Fraud case triage with a local LLM
- Phishing email screening with a local LLM
- Log and alert triage with a local LLM
- Checking text for personal data with yes/no questions
- Prompt injection screening with a small model
- Document classification with a local LLM
- Product categorization with yes/no questions
- Contract clause detection with a local LLM
- Refund request triage with a local LLM
- Detecting cancellation intent in customer messages
- Reranking search results with yes/no questions
- RAG evaluation with yes/no questions
- RAG faithfulness check with a local LLM
- Hallucination detection with a local LLM
- LLM regression tests in CI with yes/no checks
- Rubric design for an LLM judge
- Pairwise comparison with a yes/no judge
- LLM judge bias and how to control it
- Evaluation metrics for yes/no classifiers
- Building a yes/no test set for your own data
- Accuracy by kind of question: why one number hides failures
- Generating test questions with answers computed by code
- Benchmark contamination and truly held-out tests
- An LLM router with yes/no questions
- A model cascade: small model first, large model on doubt
- Semantic routing vs yes/no questions
- Gating AI agent tool calls with yes/no checks
- AI agent guardrails with yes/no questions
- Stop conditions for AI agents
- Logging LLM decisions for audit
- Reducing LLM cost with local yes/no decisions
- Replacing chat LLM calls with yes/no questions
- Structured output vs a probability
- A Python client for local LLM decisions
- Calling a local LLM decision server from JavaScript
- Local LLM yes/no decisions in n8n
- A Slack bot that uses local LLM decisions
- Home Assistant automations with local LLM decisions
- A LangChain tool for local yes/no decisions
- Batch decisions from files with jev decide
- Running LLM yes/no checks in GitHub Actions
- Securing a local LLM server with an API key
- curl examples for a local LLM decision API
- Self-hosted AI for decisions
- A private LLM for text classification
- On-premise LLM for business decisions
- GDPR and automated decision-making with an LLM
- Offline AI for decisions: no network needed
- Edge AI decisions on a CPU
- Run an LLM locally without a GPU
- Small language models explained
- When a small model is enough, and when it is not
- An LLM on a laptop: what it can do in real time
- What is GGUF, for someone deploying a classifier
- GGUF quantization types explained: Q4_K_M, Q8_0 and others
- GGUF vs safetensors
- llama.cpp vs Ollama for a classification service
- llama-cpp-python vs calling llama.cpp through ctypes
- llama.cpp on Windows without compiling
- Running llama.cpp CPU only
- Using llama.cpp prebuilt binaries instead of building
- Download a GGUF model from Hugging Face