Repository navigation
stuntd 0.2.0
What's new
OpenAI Decisions API. stuntd now answers POST /v1/decisions, the typed-question API OpenAI opened in beta on 2026-10-06 (gpt-6-luna). Each predicate, choice and score question becomes a decision site exactly like a Jev question, so recording, training, report, the novelty gate and the modes all work as before:
- In front of OpenAI, a request is answered locally only when every question has a live head that is sure; otherwise it goes to OpenAI untouched and each answer becomes a label. Refusals are relayed and never recorded.
- With no upstream, heads answer what they know and zero-shot Laya answers the rest, in the Decisions shape the
openaiSDK parses (boolean choice values stay booleans, scores are the probability-weighted level). - Not learned, relayed whole: images, colliding choice values, more than 10 score levels and the other cases listed in the README.
Not measured against the live API: the shapes were checked against the Decisions guide and the openai 3.27.0 SDK types, and the release check ran the real SDK against stuntd in both modes.
Stock sentence encoders. training.encoder can name intfloat/multilingual-e5-base, intfloat/multilingual-e5-small, sentence-transformers/all-MiniLM-L6-v2 or another sentence encoder, and stuntd train fits a linear head on its frozen pooled vectors (one vector per row, a few seconds per site). Laya stays the default and the only zero-shot model.
- Encoders load with
trust_remote_code=Falseand only from an allowlist of architectures. - Every head records the encoder it was trained on. A mismatched head is never served; it shows as
encoder-mismatchuntil you retrain.
On MASSIVE intent (60 intents, 3,000 training rows, the full official test split), accuracy of every answer:
| Laya | multilingual-e5-base | multilingual-e5-small | all-MiniLM-L6-v2 | |
|---|---|---|---|---|
| ru | 26.2% to 27.2% | 81.6% | 78.6% | 54.7% |
| de | 29.5% to 31.5% | 79.6% | 75.9% | 65.1% |
| es | 34.3% to 34.4% | 80.5% | 78.0% | 63.5% |
| zh | 42.7% to 44.1% | 79.8% | 78.3% | 40.2% |
| ja | 32.1% to 32.2% | 79.8% | 79.5% | 57.7% |
| en | 60.1% to 60.7% | 83.7% | 81.2% | 81.0% |
| Banking77 | 65.1% to 66.1% | 89.6% | 87.5% | 90.0% |
multilingual-e5-base answers 18.0% to 41.9% of the MASSIVE test rows locally at 97.9% to 99.3% accuracy, and decides in about 14 to 27 ms on a laptop CPU against 340 to 400 ms for Laya. Laya's numbers vary between two runs, so its cells give both. benchmarks/encoders.py reproduces every number, and benchmarks/README.md has the full tables, hardware and dataset revisions.
Jev-compatible teachers. The README now documents Kev, Mica v0.1 4B, Perplexity Decider v1.1 and Strom as jev.upstream: how to run or reach each one, and what differs from Jev. tests/test_teachers.py contract-tests each one's documented request and response through stuntd. Nothing was run against the real models.
The new jev.path setting (default /v1/systemone) covers Perplexity's hosted /v1/decisions path.
Fixes. A body nested too deeply or carrying a lone UTF-16 surrogate no longer gives a 500 on /v1/systemone; it is relayed unlearned in proxy mode and gets a 400 locally.
Behaviour changes to know
/v1/decisionsused to be relayed to the upstream like any other path; it is now learned. Withupstreamempty it is answered locally, so a daemon with no top-level upstream now loads the Laya checkpoint at start even whenjev.upstreamis set./healthzstill counts a site with an encoder mismatch as live;stuntd statusshows the real state.