OpenJev

Fast, calibrated, typed decisions from an open model. OpenJev is an open-source "System One" decision server. Send it a state and a set of typed questions (yes/no, choice, score). It returns a probability and a confidence for every answer in tens of milliseconds. It reads the answers straight off the model's probabilities. It generates no text and parses nothing, so an answer cannot go off-schema. Questions can also ask about images.

OpenJev speaks the same wire API as TypeSafe's Jev, so their SDKs work against it unchanged. It runs DiffusionGemma 26B-A4B (Apache-2.0) two ways: through vLLM on an NVIDIA GPU, or in-process through MLX on Apple silicon. Both backends also serve text generation on an OpenAI-compatible /v1/chat/completions, at no extra GPU memory.

Hosted for free on Codiv, an inference platform for open System One models: sign up and get 100M input tokens, no card required. https://api.codiv.ai/v1/systemone

OpenJev is an independent project. It is not affiliated with or endorsed by TypeSafe AI.

pip install typesafe-sdk
export TYPESAFE_BASE_URL=https://api.codiv.ai   # or http://127.0.0.1:8080 for your own server
export TYPESAFE_API_KEY=sk-codiv-...

from typesafe_sdk import TypeSafeClient

client = TypeSafeClient()
r = client.system_one(
    "Everything is down and we have a demo with our biggest client at noon.",
    {
        "urgent": {"type": "noul", "instructions": "Does the customer need a reply within the hour?"},
        "team":   {"type": "choice", "instructions": "Which team should handle it?",
                   "criteria": {"outage": "service down", "billing": "charges, refunds", "feature": "requests, how-to"}},
        "tone":   {"type": "score", "instructions": "How upset is the customer?",
                   "criteria": ["calm", "annoyed", "furious"]},
    },
)
r.nouls["urgent"].noul        # 1.00
r.choices["team"].choice      # "outage", confidence 1.00
r.scores["tone"].score        # 2.00 (expected level, 0-indexed)

Or with curl:

curl https://api.codiv.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "openjev-latest", "state": "I was charged twice this month.",
       "questions": {"is_billing": {"type": "noul", "instructions": "Is this a billing issue?"}}}'

Question types:

• noul (yes/no): takes optional criteria: {true, false} and returns {noul: P(yes)}.
• choice: takes criteria: {name: description} and returns {choice, probabilities, confidence}.
• score: takes criteria: [level0, level1, …] (2–10 levels) and returns {score: Σ i·pᵢ, legend, probabilities, confidence}.

confidence is 1 − H(p)/ln K. It is 1 when the model is certain and 0 when the distribution is uniform. usage.input_tokens counts prompt tokens, including image tokens. usage.output_tokens is 0 unless you set think (below).

Every response carries a Server-Timing header that splits where the time went:

server-timing: model;dur=41.2, server;dur=2.8, total;dur=44.0

model is time spent waiting on the model, summed over the request's reads, so it can exceed total when reads run in parallel. server is the rest of the wall clock: compiling the schema, tokenizing, validating and serializing. Neither includes your network.

Errors follow the same shapes as Jev, checked against the live API:

• 422 with a FastAPI validation list for a field of the wrong shape.
• 400 with the reason as plain text for a question the server cannot ask (no options, too many options, or too many score levels).
• 400 api_usage_error for an unknown model or question type.
• {"detail": {"error_type", "message"}} for auth errors (401/403).
• 429 for rate limits, and 529 when the server is overloaded.

Known differences from Jev:

• Model names are OpenJev's own. jev-latest and jev-preview are aliases. A pinned Jev version such as jev-1.13.0 answers 400 Unknown model.
• A choice with one option, or a score with one level, gets a direct answer (probability 1) with no read, so it bills no tokens. Jev returns the same values and bills for the read.
• A body nested a thousand levels deep answers 422. Jev answers 500.
• The server answers many questions in chunks of about 12 per read, still in parallel.

These optional request fields are OpenJev additions to Jev's contract. Leave them out and a request behaves exactly like Jev. TypeSafe's SDKs never send them. They come from the example server in vllm-project/vllm#57250.

curl https://api.codiv.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "openjev-latest", "state": "Look at the photo.",
       "images": ["data:image/jpeg;base64,/9j/4AAQ..."],
       "questions": {"hotdog": {"type": "noul", "instructions": "The photo shows a hot dog"}}}'

think and sequential need a text state, so you cannot combine them with images. Such a request gets a 400.

POST /v1/chat/completions generates text with the same DiffusionGemma, OpenAI style. Tools that talk to a chat model can point their base URL at OpenJev. Use model diffusiongemma-26b. Both backends serve it, streaming and not, and a client cannot tell them apart except where this section says so.

vLLM refuses some fields for diffusion models, so OpenJev adjusts requests instead of failing them:

• It ignores temperature, seed, min_p, logit_bias, penalties and reasoning.
• response_format (json_object or json_schema) becomes an instruction to reply with JSON only. The server returns the first JSON object in the reply as the content.
• max_tokens defaults to 1024, with a cap of 8192. Thinking stays off unless you set chat_template_kwargs: {"enable_thinking": true}. The thought then comes back separately in message.reasoning, never in content.
• stream: true streams server-sent events and always ends with a usage chunk.
• The endpoint supports tools (tools, tool_choice).

On MLX, three of these differ. tools, tool_choice, logprobs and top_logprobs are accepted and ignored, not answered. A stop string that is more than one token is dropped, because only a single token can end a denoised block. And there is no message.reasoning: the model's thought channel is stripped from the reply, so enable_thinking changes the answer but does not return the thought.

Generation denoises the output in 64-token blocks, so it costs far more GPU time than a System One read. At most 8 generations run at once, so they never crowd out reads. On MLX the model runs on one thread, so generations queue rather than overlap, and a client that disconnects stops the reply at the next block instead of the next token.

from openai import OpenAI

client = OpenAI(base_url="https://api.codiv.ai/v1", api_key="sk-codiv-...")
r = client.chat.completions.create(
    model="diffusiongemma-26b",
    messages=[{"role": "user", "content": "Summarize: 3 nonstop flights, cheapest $212 on Delta."}],
)
print(r.choices[0].message.content)

DiffusionGemma is a discrete diffusion model. It denoises a whole canvas of tokens per forward pass instead of generating left to right. OpenJev uses that to read answers instead of writing them.

It builds a canvas where the only masked tokens are the answer slots, one token per question:

canvas in                 one read-only pass         answer out
  q1: [?]        ──►      P(yes) 0.001        ──►    noul  0.001
  q2: [?]                 P(A) 0.000                 choice "billing"
                          P(B) 0.999                 confidence 0.997
                          P(C) 0.000
  q3: [?]                 P(0) 0.000                 score 1.00
                          P(1) 0.996
                          P(2) 0.004

Every label is a single token: yes/no for a noul, A/B/C for a choice, 0/1/2 for a score. The model never writes into those slots. One read-only pass returns the probability distribution over each slot, and that distribution is the answer. The numbers above are a real read of "The invoice looks wrong again. Second time this quarter.": not urgent, billing, mildly annoyed.

Two consequences follow. An answer cannot go off-schema, because the read scores only the label tokens. And the confidence comes from the model's own distribution, not from a number the model reports about itself.

If any slot is uncertain (entropy > 0.1), OpenJev reads three more times with fresh noise and averages the four results. The policy is all or nothing: one read, or four. One uncertain question causes a re-read of all the questions in the request. The extra reads are the server's policy, so they add no tokens to usage. Set samples: 1 to pin a single read. Question ids never reach the model: it sees q1, q2, q3.

The vLLM side of this is vllm-project/vllm#57250, merged on 2026-09-22, which adds seeded canvases, read-only steps, step caps and pinned canvas positions for DiffusionGemma. openjev/engine.py is adapted from that PR's structured_server.py example. It adds async I/O, bounded concurrency and backpressure.

Two backends serve the same /v1/systemone. Pick by hardware:

You need an NVIDIA GPU with at least 24 GB of memory for the NVFP4 checkpoint (tested on an RTX PRO 6000 Blackwell, sm_120).

A prebuilt image is on Docker Hub, so there is nothing to compile. razorback16/openjev runs vLLM's DiffusionGemma structured reads in one container with the Jev-compatible API server, on CUDA 13. It pins the upstream vLLM commit shown below and makes the two changes listed under Caveats.

git clone https://github.com/razorback16/openjev && cd openjev
docker compose up -d          # OpenJev on 127.0.0.1:8080 once the model has loaded
curl localhost:8080/v1/models

Or without compose:

docker run -d --gpus all --ipc=host -p 127.0.0.1:8080:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface razorback16/openjev:0.4.0

The model weights (about 18 GB) download on first start into ~/.cache/huggingface. Use docker compose build to build the image yourself instead. vLLM listens only inside the container. Set OPENJEV_UPSTREAM to skip it and use a vLLM server you already run.

Measured on an RTX PRO 6000 using 38% of the GPU, with 3 questions per request and cache-busted states:

One request at a time, on the same GPU, with samples: 1:

Without Docker:

git clone https://github.com/vllm-project/vllm && cd vllm
VLLM_COMMIT=1b3b88ec2b7457aa030db4d0e7d8aaf04f6d0fb8   # the commit the image pins
git checkout $VLLM_COMMIT
# a choice of more than 128 options needs a larger cap, as in the image
sed -i 's/^MAX_LOGPROB_TOKEN_IDS = 128$/MAX_LOGPROB_TOKEN_IDS = 512/' vllm/sampling_params.py
VLLM_USE_PRECOMPILED=1 VLLM_PRECOMPILED_WHEEL_COMMIT=$VLLM_COMMIT pip install -e .
vllm serve nvidia/diffusiongemma-26B-A4B-it-NVFP4 --served-model-name dgemma \
  --diffusion-config '{"canvas_length": 64}' --max-logprobs 32 --enable-prefix-caching \
  --async-scheduling --attention-backend TRITON_ATTN \
  --limit-mm-per-prompt '{"image": 8, "video": 0}' \
  --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 \
  --override-generation-config '{"max_new_tokens": null}'
pip install -e path/to/openjev && python -m openjev

This build reads and generates exactly as the image does, but it prefills images causally: the image's other change, bidirectional attention inside each image, is a patch to a vLLM source file (see Caveats). Apply docker/patches/vision_prefix_lm.py to the checkout above to match the image.

A Mac needs no vLLM and no Docker. OPENJEV_BACKEND=mlx runs DiffusionGemma inside the OpenJev process through MLX and mlx-vlm. The 4-bit weights need about 16 GB of memory.

pip install -e '.[mlx]'
OPENJEV_BACKEND=mlx python -m openjev     # 127.0.0.1:8080

/v1/systemone answers reads with the same prompts, canvases and seeds as the vLLM backend, including images, samples, sequential, steps and the automatic re-reads. Each denoise step past the first reuses the one prefill of the prompt and writes back only the answer slots, so the template cannot drift: more steps cost GPU time, not prompt tokens. On vLLM the same holds because a multi-step read pins every canvas position but the answer slots. think writes the thought with mlx-vlm's own denoise loop, then reads after it, and bills exactly as vLLM does.

An image read builds its prompt with the mlx-vlm processor, which expands each image into its soft tokens. usage.input_tokens and OPENJEV_MLX_MAX_PROMPT both count that expanded prompt. The server keeps the prefill of a recent prompt, so the re-reads and the samples of one request share a single vision pass.

Reads run one at a time, so this backend suits local use rather than serving. A 3-question request takes about 0.2–0.4 s on an M3 Ultra and about 0.39 s on an M4 Max, both with the 4-bit weights. 16 concurrent requests finish at about 4 req/s.

OPENJEV_MLX_TEST_MODEL=path/to/weights pytest tests/test_mlx_model.py   # tests against the real model

The server reads its settings from the environment.

• OpenJev pins upstream vLLM rather than a fork, and makes two changes to that commit when it builds the image. Both are checked at build time, so a moved anchor fails the build instead of shipping silently:
  • Upstream allows 128 exact label ids per request; a 255-option choice needs more, so the build raises the cap to 512.
  • The checkpoint's config asks for bidirectional attention inside each image (use_bidirectional_attention: "vision"). vLLM applies that for Gemma4 but not for DiffusionGemma, whose prepare_attn never passes the image ranges, so image tokens would be prefilled causally. docker/patches/vision_prefix_lm.py passes them, reusing vLLM's own compute_mm_prefix_ranges. An upstream fix would make it unnecessary.
• Answer quality is the quality of DiffusionGemma 26B-A4B used in this mode. Evaluate it on your own tasks before you rely on it.

pip install -e '.[test]' && pytest

Apache-2.0. The DiffusionGemma weights are Apache-2.0 (NVIDIA / Google).