Skip to content

Jev

Jump to bottom
Diego Amicabile edited this page Sep 29, 2026 · 1 revision

Living page: the reference for Jev, the typed-decision model this project calls through OpenRouter. Written from the vendors' documentation and from the user's own working client, read on 2026-09-29. Every claim carries its source in square brackets (the list is at the end), or says "not documented". Nothing here was measured by a call from this project.

Jev

Jev is a model from TypeSafe AI that answers typed questions about some text and returns a probability. It writes no text and no explanation [S1, S6]. The project calls it with scripts/jev-ask.ps1. When to use it, and when never, is in the operating guide (§2.1 Router row, §3 Jev preference). This page holds the facts behind that choice.

What it is

  • TypeSafe calls it a System One model: fast, structured decisions for software. You send a state (the text to judge) and named questions; you get back one typed answer per question, with probabilities [S6].
  • It is trained to give calibrated probabilities. TypeSafe says calibration is measured across groups of answers and does not guarantee that any one answer is right [S6]. It does not write replies, code or explanations [S6].
  • TypeSafe's announcement (Sep 15, 2026) claims answers in 70 ms to 500 ms, and a price of $0.042 per million input tokens with free output [S1]. Those are the vendor's own claims.

Model id, and what is newer

Item Value Source
OpenRouter model id typesafe/jev-1.13 (pinned) [S3, S4]
Dated snapshot that answers typesafe/jev-1.13-20260917, reported in the response's model field [S4, S5]
Alias that moves ~typesafe/jev-latest, currently an alias of typesafe/jev-1.13 [S3]
Bare ids jev-1.13 and jev-latest are accepted and mapped onto the typesafe/ namespace on the System One endpoint [S4]
TypeSafe's own id jev-1.13.0; aliases jev-latest and jev-preview both point to it [S7]
Released September 18, 2026 (OpenRouter's model page) [S2]
  • The model is missing from the default model list. GET https://openrouter.ai/api/v1/models returns text-output models, and Jev's output type is "decisions", so it is absent (and absent from OpenCode's OpenRouter list for the same reason). GET https://openrouter.ai/api/v1/models?output_modalities=all (or =decisions) lists it [S3]. This is my reading of the two responses, not a sentence in the documentation.
  • Nothing newer than jev-1.13 is documented. jev-latest points at 1.13, and jev-preview points at the same model as jev-latest ("There is no preview build available right now") [S3, S7]. A model named typesafe/jev-router also exists, but it is a text-output router that "runs on Jev", not a decision model [S3]. Other decision-output models are on OpenRouter (upstage/solar-decide, respan/span-01, jaredpalmer/kev-4b); this page does not assess them [S3].
  • The pinned id is the one to use for thresholds: TypeSafe says an alias moves when a release ships and that thresholds tuned on one version should pin its id [S7].

Endpoint, headers and request

Two OpenRouter routes take the same body:

  • POST https://openrouter.ai/api/v1/systemone, the System One route, compatible with TypeSafe's SDKs; the SDK appends /v1/systemone to the base URL https://openrouter.ai/api [S4, S8]. The user's client uses this one [S9].
  • POST https://openrouter.ai/api/alpha/decisions, the alpha Decisions route, which OpenRouter's own tutorial uses [S10].

Both answer an unauthenticated request with HTTP 401 (checked with a key-less POST on 2026-09-29), so both exist. Which is steadier is not documented; the alpha route is labelled alpha. Sources disagree on nothing else about the request.

TypeSafe's direct route is POST https://api.typesafe.ai/v1/systemone with TYPESAFE_API_KEY [S7, S11]. TypeSafe's own signups were paused on 2026-09-26, which is why the user's client goes through OpenRouter [S9].

  • Headers: Authorization: Bearer <OpenRouter key> and Content-Type: application/json [S4, S10]. Other headers (OpenRouter's attribution headers, for example): not documented for this route. A request may carry an x-session-id header or a session_id body field, user, trace and a provider preferences object; none is needed [S4].
  • Body: model, state and questions are required [S4].
  • state is a string, a JSON object or a JSON array, text only [S4, S12].
  • questions is a map from a name you choose to a question. The name is not sent to the model [S11].

A minimal request (the shape is from the OpenRouter and TypeSafe references [S4, S11]; the values are mine, and no call was made with it):

{
  "model": "typesafe/jev-1.13",
  "state": "Bug: the army panel's order buttons show no text.",
  "questions": {
    "breaks_play": {
      "type": "noul",
      "instructions": "Does this bug stop a player from playing?"
    }
  }
}

The response has this shape (fields from the OpenRouter reference and its example [S4]; the numbers are illustrative):

{
  "id": "gen-dec-...",
  "model": "typesafe/jev-1.13-20260917",
  "provider": "TypeSafe",
  "answers": {
    "breaks_play": { "type": "noul", "noul": 0.93 }
  },
  "usage": { "input_tokens": 60, "output_tokens": 20, "cost": 0.0000025 }
}

The three question types

Every question has type and instructions (a string, or an object or array of structured guidance) [S11]. All questions in one request see the same state, are answered in parallel and cannot see each other's answers [S10].

Type Question extra Answer Source
noul (yes or no) optional criteria: {"true": ..., "false": ...} {"type": "noul", "noul": p}, where p is the probability of yes, from 0 to 1 [S11]
choice required criteria: a map from each option to its description (or null) {"type": "choice", "choice": "<option>", "probabilities": {option: p, ...}, "confidence": c}; the probabilities sum to 1 [S11]
score required criteria: an ordered array of level descriptions {"type": "score", "score": x, "probabilities": {"0": p, ...}, "legend": {"0": "<level text>", ...}, "confidence": c}; x is the probability-weighted level, index 0 being the first criterion [S11]
  • A Noul has no confidence field. A Choice's or Score's confidence is derived from how concentrated its distribution is [S11, S13].
  • Do not compare a Noul with a Choice. TypeSafe warns that a threshold tuned on a Noul should not be carried to a Choice, and that a question and its negation as two Nouls need not sum to 1 [S14].
  • The OpenRouter reference describes the Noul criteria as optional; the user's client sends only instructions [S4, S9].

Limits

Limit Value Source
Options in one Choice at most 255 [S11]
Levels in one Score at least 2; at most 10 [S11]
Questions in one request no number documented; TypeSafe advises putting every question you need into one request, and its cookbook batches 13 [S15, S16]
Text per request 64k tokens for the state plus all questions; 32k tokens for the state plus the longest single question [S7]
Context length on OpenRouter 32,000 tokens (max_completion_tokens is listed as 28,800) [S3]
Request body size in bytes not documented (OpenRouter lists a 413 Payload Too Large error) [S4]
Input types text only: string, object or array of text. No images, audio or video [S12]

When the state is too big. What the API returns for a state over the limit is not documented. The error tables list 400, 401, 402, 403, 404, 413, 429 and 500 on OpenRouter [S4], and 401, 422, 429 and 529 at TypeSafe [S11]. No tokenizer is documented either. Size matters before the limit too: accuracy falls as the state fills with detail the question does not need, and TypeSafe's advice is to filter first and send only what the question needs [S14].

For a 2,000-line diff, my estimate (not a documented figure) is 20,000 to 40,000 tokens, which can pass the 32k line. scripts/jev-ask.ps1 therefore does not send an over-limit state: it estimates 4 characters per token and refuses a state over 96,000 characters (about 24k tokens) with a one-line reason, and it truncates nothing. Send the diffstat, or only the hunks the question is about.

What Jev is weak at, from TypeSafe's list for jev-1.13 (reviewed 2026-09-17) [S14]: reading a question literally rather than by intent; arithmetic and counting; comparing dates (it reads them as text); questions that need several hops; a large state with irrelevant detail; content written to steer it (the state is data, and injected instructions can move the answer); and generating text. English is the primary language [S7]. The user's client found dates helped its task, against this advice [S9]; test on the real task.

Rate limits and price

  • Rate limit (TypeSafe): 250,000 tokens per second and 1,200 requests per minute, over which a request returns 429; TypeSafe says the limits "are adjusting dynamically" and can change without notice [S7]. Whether these limits apply to traffic that arrives through OpenRouter is not documented. OpenRouter documents no Jev-specific limit; its own limits are free-model caps, protection against floods and an in-flight spending budget [S17].
  • Price as OpenRouter bills it: $0.042 per million input tokens, $0 for output tokens [S2, S3]. OpenRouter charges "at the input price shown on the Jev model page", and each response reports the call's cost in USD in usage.cost [S10]. One example response bills 476 input tokens at $0.000019992 [S4], which is exactly 476 × $0.042 / 1,000,000. Any OpenRouter fee on top of the model price: not documented on these pages.
  • Speed: 0.4 to 0.8 seconds per call in the user's runs [S9]. TypeSafe claims 70 to 500 ms [S1].

What newscollection2027 found

The user's client is nc bench-jev (PR #85) and nc judge --backend jev (PR #87) in diegoami/newscollection2027. It asks one Noul, "do these two articles report the same event", about a pair of news articles. It pins jev-1.13, calls OpenRouter's System One route with the standard library only, times out after 30 seconds, and retries 429, 5xx and timeouts up to four attempts [S9, S18].

  • Key variable: OPENROUTER_API_KEY (set in config/judge.yaml as jev.api_key_env). Without it, the pipeline step prints "skipped" and does nothing [S9, S18]. To go direct to TypeSafe it would change the base URL and use TYPESAFE_API_KEY [S18].
  • Thresholds: act on a yes at 0.9 or above; discard below 0.1; give everything between to the Claude judge (auto_yes_at and prefilter_below in the same file) [S9, S18].
  • Item count: 280 human-labelled pairs, 117 of them matches (2026-09-26). The cutoff was picked on one half of the labels and scored on the other, each half with about 55 matches, so one or two pairs move precision by 0.02 to 0.04 [S9].
  • Accuracy at the ends (with dates): pairs Jev scored below 0.1 were 2% matches, and pairs at 0.9 or above were 97% matches (without dates: 3% and 100%). The middle was rough: the 0.1 to 0.3 bin was 50% matches. At a 0.9 cutoff, held out: precision 0.964, recall 0.509. At 0.6, the cutoff the rule picked: precision 0.891, recall 0.925, against the Claude judge's 0.931 and 0.870 on the 104 pairs both answered. Verdict of the trial: not yet good enough to replace the judge; good as a pre-filter [S9].
  • The three-way split, on the labels (queue range, with dates): 42 pairs in Jev's no lane (3 matches lost, all three look like label errors), 79 for the judge, 59 in the yes lane (2 wrong links) [S9].
  • On the live queue (2026-09-26): of 1,059 unjudged pairs, 31 confident yeses (30 agreed with the Claude judge) and 40 confident nos (all 40 agreed); the owner agreed with 31 of 31 confident yeses in the tail the judge had not reached [S18]. Wired into the pipeline, the judge's queue fell from 906 to 346 pairs [S18].
  • Cost: 560 calls for $0.015 in the trial; the live-queue run of 1,059 pairs cost $0.03 [S9].

The measurements are on a different task from ours (news pairs, not bug reports or sweep tables). They support the 0.9 and 0.1 split; they do not measure our questions.

What OpenRouter and TypeSafe say about keeping request bodies

Our states derive from private evidence and leave for OpenRouter and TypeSafe. That is the user's decision (operating guide §3). What the sources say:

  • OpenRouter itself: it does not store prompts or responses unless you turn on Private Input and Output Logging, or "OpenRouter use of inputs/outputs" (a 1% discount for letting OpenRouter use them); both are off by default. It always stores request metadata (token counts, latency), not content [S19]. A small sample of prompts is categorised anonymously by a model with zero data retention [S19].
  • OpenRouter's record of the TypeSafe provider: training: false, retainsPrompts: false, canPublish: false [S3, S20]. The endpoint TypeSafe | typesafe/jev-1.13-20260917 is on OpenRouter's public list of zero-data-retention endpoints (GET https://openrouter.ai/api/v1/endpoints/zdr), so a request that demands ZDR (provider: {"zdr": true}) can still reach it [S21, S4]. scripts/jev-ask.ps1 does not send that field.
  • TypeSafe's models page: "Jev is not trained on customer requests or responses"; zero data retention is offered "for enterprise customers" (contact sales) [S7, S22].
  • TypeSafe's privacy policy: it will not train or fine-tune models on your input, and will not disclose input to a third party other than its service providers. It keeps personal data "for as long as reasonably necessary" [S23]. Its data processing agreement says customer personal data is kept "for as long as necessary" for the purpose of processing [S24].
  • A tension that is not resolved in the sources: OpenRouter lists the endpoint as not retaining prompts, while TypeSafe's own documents describe zero retention as an enterprise option and give an open-ended retention rule for input. Whether traffic through OpenRouter is covered by a zero-retention arrangement beyond that listing is not documented. Do not send states that must never persist at either company.

Sources

Fetched 2026-09-29.

Clone this wiki locally