🧠 LLM2Jev: Turn LLMs into Jev-Style Decision Models

LLM2Jev is an independent open-source project. It is not affiliated with or endorsed by Jev or TypeSafe.

• September 23 - MuJoCo pick-and-place demo: added LLM2Jev control of a simulated Panda arm with per-step decision.
• September 22 - Multimodal inputs: added text-and-image requests for SGLang, Transformers, and the System One HTTP API.
• September 21 - Web and Snake demos: added interactive examples for composing mixed questions and model-driven decisions.
• September 21 - Prefix reuse on cold requests: added staged candidate submission for reusing SGLang's Radix Cache, with architecture, usage, and benchmark documentation.
• September 20 - SGLang and System One API: added the SGLang scoring backend and a compatible POST /v1/systemone endpoint.

• Prefill only: compute probabilities from logits during prefill and assemble results directly, without token-by-token decoding.
• Multimodal inputs: combine text and images in state or instructions, with support for both SGLang and Transformers.
• Order-independent options: evaluate each Choice candidate independently, so reordering options does not introduce a positional preference or change their scores.
• Prefix reuse on cold requests: stage candidate submissions to reuse SGLang's Radix Cache within a single request, including a first request with no relevant cached prefix.

Candidates share state, and candidates for the same question also share its instructions. LLM2Jev first scores a real criteria candidate to establish the prefix cache, then submits candidates that can reuse it. Each candidate is scored once, reducing repeated computation for long inputs with many candidates.

Learn how it works: From Jev Request to LLM Request → Shared-prefix design.

On Linux with a supported NVIDIA GPU, run a local model through SGLang:

git clone https://github.com/Yinsongxu/LLM2Jev.git
cd LLM2Jev
uv sync --extra sglang
source .venv/bin/activate
python examples/sglang_inference.py --model-path /path/to/model

The example submits Choice, Score, and Noul questions and prints the response as JSON. Replace /path/to/model with a local Hugging Face-compatible causal language model directory.

See Installation for environment requirements, SGLang and Transformers dependencies, and uv or pip installation.

See the Usage guide for complete examples:

• SGLang Python API
• Transformers backend
• System One HTTP API
• Choosing between staged and all
• Multimodal inputs

Web demo	Snake demo
MuJoCo pick-and-place demo

See Performance benchmarks for the Qwen3-1.7B / RTX 5090 measurements, test conditions, and comparison of staged and all across cold and warm caches. Gains depend on input length, candidate count, and cache state.

• More benchmarks across model sizes, datasets, and workloads, covering decision quality, latency, and throughput.
• An interactive web demo for submitting questions and inspecting probabilities.
• Initial local-image support for Transformers and SGLang.
• More multimodal tasks and demos.

python -m unittest discover -s tests -v

This project is licensed under the Apache License 2.0.