-
Notifications
You must be signed in to change notification settings - Fork 129
curl examples for a local llm api
To POST JSON with curl, send the body with -d and a Content-Type: application/json header,
or use --json, which sets that header and Accept: application/json for you and implies
POST. Against a local jev serve that is one command to http://127.0.0.1:8017/v1/systemone,
and the answer comes back as JSON with one noul, the probability of yes, per question. Add
-D - to print the response headers, including Server-Timing, the server's own inference and
total time.
curl is the fastest way to find out whether a problem is in the model, the server or your client. If the same request works from curl and fails from your code, the model is not the problem.
This page is the basic request, bodies from files, auth, timing, the other endpoints and what a
422 looks like. Each command is a minimal sketch to adapt; curl facts are from its manual,
fetched 2026-09-29.
This is the README's Quickstart:
curl http://127.0.0.1:8017/v1/systemone -H 'Content-Type: application/json' -d '{
"model": "jev-latest",
"state": "I was charged twice for the same order.",
"questions": {"billing": {"type": "noul", "instructions": "Is this a billing problem?"}}}'{
"model": "jevos-v2",
"answers": {"billing": {"type": "noul", "noul": 0.94}},
"usage": {"input_tokens": 27, "output_tokens": 0}
}The -H matters. curl's manual says -d sends data with the POST method using the content type
application/x-www-form-urlencoded, so without the header the server is told the body is a
form, not JSON. Since
curl 7.82.0, --json '<body>' does both jobs: it "adds Content-Type: application/json and
Accept: application/json headers" and "implies POST unless a different method is set". The
model field accepts jev-latest, any other jev-* name, or the served model's name; the
answer always names the model that answered.
Long JSON on a command line is fragile, and quoting rules differ between shells, Windows shells in particular. Put the body in a file and let curl read it:
curl http://127.0.0.1:8017/v1/systemone -H 'Content-Type: application/json' \
--data-binary @request.jsonThe manual describes --data-binary as posting data "exactly as specified with no extra
processing whatsoever", with @ followed by a filename to read from and @- for stdin. The same
request.json can be answered without any server by jev decide, which reads the identical
body; that route is on batch decisions from files with jev decide.
Writing the questions inside the file well is its own subject, covered by
how to write yes/no questions an LLM answers well.
If the server was started with JEV_API_KEY, every call except /health needs the key:
curl http://127.0.0.1:8017/v1/systemone -H "Authorization: Bearer $JEV_API_KEY" \
-H 'Content-Type: application/json' --data-binary @request.jsonWithout it the response is 401 with WWW-Authenticate: Bearer. Double quotes around the header
let the shell expand the variable; single quotes would send the literal text $JEV_API_KEY. Setup
and what stays open is on securing a local LLM server with an API key.
-D - writes the received headers to stdout; -s keeps the progress meter out of the way, and
-o /dev/null drops the body when you only want the headers:
curl -s -D - -o /dev/null http://127.0.0.1:8017/v1/systemone \
-H 'Content-Type: application/json' --data-binary @request.json -w 'wall: %{time_total}s\n'Every successful response to /v1/systemone carries Server-Timing: ..., inference;dur=..., total;dur=..., in
milliseconds: inference is the model's work, total is everything inside the engine,
including any wait for other requests; small requests that arrive at the same time are read
together in one model call.
%{time_total} is curl's wall-clock time for the whole transfer, in seconds. The gap between
the two is the HTTP layer and the connection, which on 127.0.0.1 should be small and over a
network is not. For honest numbers, repeat the call, discard the first runs and take the median
and p90, as described on
measuring LLM latency: median, p90 and warm-up. On the
reference laptop the README gives 26 ms for a 30-token request and 112 ms for 191 tokens.
curl -s http://127.0.0.1:8017/health
curl -s http://127.0.0.1:8017/v1/models -H "Authorization: Bearer $JEV_API_KEY"/health returns {"status": "ready", ...} once the model is loaded, with the served model's
name and engine metadata; it never needs a key. Before the model is loaded the port does not accept connections at all, so in a
start-up script, loop on /health until it answers. /v1/models lists the served model and
its jev-latest alias. The Authorization header is only needed when a key is set.
Send a model name the server does not serve:
curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:8017/v1/systemone --json '{
"model": "my-model", "state": "test",
"questions": {"q": {"type": "noul", "instructions": "Is this a test?"}}}'The status is 422 and the body is a validation error, with loc pointing at the field. For this request loc is ["body", "model"] and the message says that
the server answers as its served model and accepts any jev-* alias. Other requests that get a
422: an unknown or misspelt field, a choice with one option or a score with one level,
an empty state, or a question whose prompt, the state plus that question, is longer than the
context limit.
In scripts, --fail-with-body makes curl return error 22 on HTTP 400 and above while still
showing the body, so a shell && chain stops on a 422 and you can still read why. Add -S
with -s to see curl's own error message when a connection fails.
How do I POST JSON with curl? curl URL -H 'Content-Type: application/json' -d '<json>',
or curl URL --json '<json>' with curl 7.82.0 or later.
How do I send a JSON file with curl? --data-binary @file.json with the Content-Type
header; @ followed by a filename tells curl to read the data from that file.
How do I see response headers with curl? -D - prints them to stdout; -i includes them
in the output before the body.
Why does the server reject my curl request with 422? Read loc in the body: usually the
model name, a misspelt field, or a choice with one option or a score with one level.
Why does my request fail with 401? The server was started with JEV_API_KEY; add
-H "Authorization: Bearer <key>".
See also: ask a local LLM a yes/no question and get P(yes), a Python client for local LLM decisions and calling a local LLM decision server from JavaScript.
- The Quickstart request and answer and the latencies: the jev README.
- Endpoints,
Server-Timingformat, the 401 and 422 responses and the unknown-model message: read from jev's server code. -
curl manual (
-d,--json,--data-binary,-H,-D,-i,-s,-S,-w,--fail-with-body), fetched 2026-09-29.
From the notes of jev, a yes/no decision model that runs on a laptop CPU. The first command on this page is the one in the README, and it is still the best first test.
- Ask a local LLM a yes/no question and get P(yes)
- Zero-shot text classification with yes/no questions
- LLM policy decisions: put the rule in the question
- LLM as a judge on a CPU
- Why a small LLM says yes when the answer is no
- Small LLMs and arithmetic in yes/no questions
- Our held-out benchmark said 0.855, new questions said 0.757
- jevos vs Jev vs Laya for yes/no decisions
- An open-source alternative to Jev for yes/no decisions
- jevos vs the OpenAI API for yes/no classification
- jevos vs Ollama for yes/no decisions
- jevos vs bart-large-mnli for zero-shot classification
- A yes/no LLM vs a fine-tuned BERT classifier
- jevos vs SetFit: zero-shot vs few-shot classification
- jevos vs Llama Guard for content safety checks
- jev serve vs llama.cpp server for classification
- jevos vs LM Studio: a decision server, not a chat app
- Local vs hosted LLM decisions: latency, cost, privacy
- A yes/no LLM vs a business rules engine
- LLM decisions vs keyword rules and regex
- The fastest AI model for yes/no decisions
- What makes a local LLM fast on a CPU
- Why one forward pass beats generating an answer
- Prefill vs decode: where LLM latency comes from
- Why LLM latency grows with the length of the text
- Why a hosted LLM API cannot answer in 50 ms
- Many questions about one text: why the extra ones are cheap
- CPU or GPU for a small LLM
- Latency budgets: where a 200 ms model fits
- Measuring LLM latency: median, p90 and warm-up
- Q4_K_M vs Q8_0: speed and size for a small model
- Throughput vs latency for a decision server
- What P(yes) means, and what it does not
- LLM calibration explained with yes/no answers
- Expected calibration error (ECE), explained
- Temperature scaling for LLM probabilities
- Platt scaling for a yes/no model
- Reading a reliability diagram
- How to choose a threshold for P(yes)
- Thresholds when a wrong yes costs more than a wrong no
- Human in the loop AI with a review band
- Precision and recall at a P(yes) threshold
- Base rates: why a 0.9 yes can still be wrong often
- Combining yes/no answers with AND, OR and NOT
- Logits, log-odds and P(yes)
- LLM confidence scores: probabilities vs self-reports
- How to write yes/no questions an LLM answers well
- Negation in yes/no questions for an LLM
- One condition per question: splitting compound questions
- Ask whether the text says it at all
- Scores as yes/no thresholds: is it at least high?
- Sending JSON as the text: designing the state
- Why wording changes an LLM's answer, and how to test it
- Mainly about: questions for messages with several topics
- Yes/no questions about tone and emotion
- Asking about intent: what does the writer want?
- Yes/no questions about long documents
- Using an English-only LLM with other languages
- Content moderation with a local LLM
- A Discord moderation bot with a local LLM
- Spam detection with yes/no questions
- Review moderation with a local LLM
- Email triage with a local LLM
- Support ticket routing with yes/no questions
- Urgency detection in customer messages
- Sentiment analysis with yes/no questions
- Intent detection with a local LLM
- Lead qualification with yes/no questions
- Fraud case triage with a local LLM
- Phishing email screening with a local LLM
- Log and alert triage with a local LLM
- Checking text for personal data with yes/no questions
- Prompt injection screening with a small model
- Document classification with a local LLM
- Product categorization with yes/no questions
- Contract clause detection with a local LLM
- Refund request triage with a local LLM
- Detecting cancellation intent in customer messages
- RAG evaluation with yes/no questions
- RAG faithfulness check with a local LLM
- Hallucination detection with a local LLM
- LLM regression tests in CI with yes/no checks
- Rubric design for an LLM judge
- Pairwise comparison with a yes/no judge
- LLM judge bias and how to control it
- Evaluation metrics for yes/no classifiers
- Building a yes/no test set for your own data
- Accuracy by kind of question: why one number hides failures
- Generating test questions with answers computed by code
- Benchmark contamination and truly held-out tests
- An LLM router with yes/no questions
- A model cascade: small model first, large model on doubt
- Semantic routing vs yes/no questions
- Gating AI agent tool calls with yes/no checks
- AI agent guardrails with yes/no questions
- Stop conditions for AI agents
- Logging LLM decisions for audit
- Reducing LLM cost with local yes/no decisions
- Replacing chat LLM calls with yes/no questions
- Structured output vs a probability
- A Python client for local LLM decisions
- Calling a local LLM decision server from JavaScript
- Local LLM yes/no decisions in n8n
- A Slack bot that uses local LLM decisions
- Home Assistant automations with local LLM decisions
- A LangChain tool for local yes/no decisions
- Batch decisions from files with jev decide
- Running LLM yes/no checks in GitHub Actions
- Securing a local LLM server with an API key
- curl examples for a local LLM decision API
- Self-hosted AI for decisions
- A private LLM for text classification
- On-premise LLM for business decisions
- GDPR and automated decision-making with an LLM
- Offline AI for decisions: no network needed
- Edge AI decisions on a CPU
- Run an LLM locally without a GPU
- Small language models explained
- When a small model is enough, and when it is not
- An LLM on a laptop: what it can do in real time
- What is GGUF, for someone deploying a classifier
- GGUF quantization types explained: Q4_K_M, Q8_0 and others
- GGUF vs safetensors
- llama.cpp vs Ollama for a classification service
- llama-cpp-python vs calling llama.cpp through ctypes
- llama.cpp on Windows without compiling
- Running llama.cpp CPU only
- Using llama.cpp prebuilt binaries instead of building