v0.5.1

Fixes for the fourth security audit (docs/security.md, "run 4"). No finding exposed keys, other tenants' data or the admin API.

Upgrading:

• An upstream URL with credentials (https://user:pass@host) now stops startup. httpx sent them as Basic auth in place of every caller's key. Callers bring their own keys; put a gateway's credentials in a header it adds itself.
• A [tasks."<key>"] override that the task can't take now stops startup (a confidence_floor for a yes/no task), instead of being saved and leaving the task unable to load.
• The admin CLI uses the local server's admin token only when no --url / JEVSTILLER_ADMIN_URL is given. Pass --token (or --token-file) with --url.

Fixes:

• Disk (Medium). Stored text was cut at 32,768 characters, about 128 KiB per row in 4-byte characters, and a caller with an accepted key could fill the volume with requests the student answered locally, at no cost at Jev. Stored text is now cut at 8 KiB of UTF-8, still past what any encoder reads. With less than min_free_disk_mb (1 GiB) free on the store's volume, new rows keep no text, so a volume can't fill with it.
• A confidence floor on a yes/no task was saved before it was rejected: the task stopped answering locally, and after a restart it dropped out of the index, so text retention and tenant deletion skipped its stored text. Changes are now checked before they are saved, and a task file that can't be loaded is still covered by text retention and delete-tenant.
• Admin changes and loads, deletes: a cut-off, floor, target or mode change made while the task's engine was loading was saved but never applied (a yes/no task's student kept answering at the old cut-off); one racing a delete, or the janitor's own write, could bring the deleted task back after a restart. Task changes, loads and deletes now take the same per-task lock. A change to a task deleted meanwhile is a 404.
• Admission: one caller sending new questions could flush every tenant's admission counts, so nobody's new questions became tasks. A caller now holds at most a tenth of the table.
• Forwarding: a caller's Accept-Encoding: br or zstd could come back as a still-compressed body without its Content-Encoding. The proxy now asks Jev for gzip or deflate, which it always decodes. Headers named in Connection are no longer forwarded either way, and repeated response headers (Set-Cookie) are no longer merged.
• Yes/no criteria spelled differently ({}, {"true": null}, none) are one group per request, not one each: they were counted, stored and drawn once per spelling.
• A teacher confidence that isn't a number counts as below a confidence floor in the audit, as it already did in calibration.
• Python API: Config.min_free_disk_mb (1024); Admission(max_per_caller=...) (default a tenth of max_tracked) and Admission.observe(..., caller=...).
• Website deploy: the job holding the Cloudflare token no longer runs the site's npm dependencies. The site is built in a job without secrets, and the deploy job installs only wrangler, from its own lockfile, with install scripts off. Dependabot waits 7 days before proposing a new npm release.

v0.5.0

Yes/no (noul) questions are now answered locally too, not just choice questions.

Upgrading: yes/no questions are now answered locally, by default. Until now the proxy forwarded every noul question to Jev. From this release a repeated noul question becomes a task like a choice question does, and once its student has trained and passed its checks, the proxy answers it. Read this before upgrading if your services ask noul questions:

• The default assumes your code compares the probability with 0.5. "The same answer as Jev" for a number means the same side of a cut-off, and the proxy can't see which cut-off your code uses. If it is not 0.5, set it: [manager] noul_cutoffs = [0.8] (or JEVSTILLER_NOUL_CUTOFFS=0.8), or per task with jevstiller admin cutoffs <key> 0.8. With the wrong cut-off, local answers are only guaranteed to be on Jev's side of 0.5, not of yours.

• If your code has three outcomes (no / send to review / yes), give the two edges: noul_cutoffs = [0.4, 0.6]. Requests Jev would put in the band go to Jev.

• If your code uses the probability as a number (a score, a ranking, several cut-offs), turn it off: noul_cutoffs = [] (or JEVSTILLER_NOUL_CUTOFFS=none). A local answer returns the student's own probability; only its side of the cut-off is covered. A tolerance on the value itself is planned.

• Nothing changes for choice questions, and a request that mixes question types is still answered locally only when every question in it can be.

• Going back: this release writes kind and cutoffs into each task's task.json, which 0.4.x can't read. Take a backup (jevstiller backup) before upgrading if you may need to return.

• Yes/no (noul) questions. A noul question is a two-class task trained on Jev's probability; the student answers with its own probability in Jev's shape, {"type": "noul", "noul": 0.93}.

  • The contract: you set the cut-off your code uses (noul_cutoffs, default [0.5]; at or above it is yes). target_agreement then bounds how often a local answer falls on the other side of it than Jev's, with the same calibration, shadow test and audit as for choice. Two numbers make an unsure band with three outcomes.
  • Measured on Jev's recorded answers to four yes/no questions (docs/benchmarks.md): replayed through the loop, 98% of held-out requests answered locally where Jev is decisive (a news topic, a rare intent) and 22–36% on noisy tweet questions, at 98.75–99.70% agreement with Jev's side of the cut-off. Offline over ten splits per question and eight contracts, the 2% budget held in 316 of 320 runs. Against live Jev, 71% of the last 500 of 6,000 requests were answered locally.
  • Settings: [manager] noul_cutoffs, [tasks."<key>"] cutoffs, jevstiller admin cutoffs <key> 0.8 or 0.4,0.6, POST /jevstiller/v1/tasks/{key}/cutoffs. Changing a task's cut-offs stops its local answers until a student calibrated for the new ones has passed shadow.
  • Python API: Task.noul(name, instructions, criteria=None, cutoffs=0.5); TaskManager(noul_cutoffs=...), and kind="noul" on classify / route / resolve; Jevstiller.set_cutoffs; Status.kind and .cutoffs; a teacher answers a noul task with probs={"false": 1 - p, "true": p}. JevTeacher asks Jev a Noul question for such a task.
  • A first student for a yes/no task can train while one side is still rare (rare_classes = "defer"): answers on that side go to Jev until it has enough examples.
  • experiments/record_noul.py, noul_curve.py and noul_replay.py record and reproduce the measurements.

• Docs: Coverage of Jev's API, item by item: what is answered locally (repeated choice questions on short text), what is forwarded (noul, score, mixed requests, long states past 256 tokens) and what this design cannot answer (questions that change per request). The README and benchmarks quote TypeSafe's current rate limits (80 requests/s); the JavaScript SDK is listed as untested.

• The status report no longer suggests a setting you already have. While a first student waits because fewer than two classes have enough samples, the report says so. It used to suggest rare_classes='defer', the default since 0.3.4.

• Docs: a 12-hour soak of 0.4.0 (4.3 M requests, 0 errors, RSS 300–445 MB, drift at hour 6 recovered by itself) in docs/benchmarks.md; the README's memory line quotes it.

• Docs brought up to date with 0.4.0:

  • DESIGN.md §5 and §7.5 still described the threshold and out-of-distribution rules from before 0.2.0's correction.
  • DESIGN.md §3 and the website's "When to use it" led with Jev's rate limit, although one key sustained 190 requests/s against a published 20/s (docs/benchmarks.md).
  • Also corrected: /readyz's checks, the metrics list, the environment variable for [encoder] spec, the Docker EXTRAS argument, the development setup, and a heading lost from docs/benchmarks.md.
  • DEPLOYMENT_PLAN.md now lists what came after the plan and what is next.

v0.4.0

The agreement guarantee can now cover a caller's confidence check, not just the label.

Upgrading from 0.3.x:

• Nothing changes until you set a floor. Existing tasks have none after the upgrade, and [manager] confidence_floor applies to new tasks. Set an existing task's floor with jevstiller admin floor <key> 0.6 or [tasks."<key>"] confidence_floor; it retrains once for it.

• Python API: a local answer's Result.confidence is now Jev's definition. If your code compares it with a threshold tuned on the old values, retune it. With K classes, the new value is (K·top − 1)/(K − 1) for the old value top (2 classes: 0.8 becomes 0.6). The proxy's responses are unchanged.

• Data directories from 0.3.x work as they are. 0.4.0 writes a confidence_floor field into each task's task.json and into new routing policies, and 0.3.x can't read those. To go back, restore a backup taken before the upgrade (docs/compatibility.md).

• License: Apache 2.0 from this release on. 0.3.4 and earlier stay under MIT.

• The guarantee can cover your confidence check too: confidence_floor. Jev's docs suggest treating a low-confidence answer as unsure, for example sending it to review. Until now only the label was covered.

  • The problem: the local model answers only when it's confident, so a local answer never reports a low confidence. Its confidence is its own number, and on the same request it runs lower than Jev's 80–90% of the time. With a check at 0.6, 8–37% of the requests Jev would have flagged came back confident, depending on the task. Counting a different label or a lost flag, the caller's outcome changed on more than 2% of requests in 20 of 25 benchmark splits, while the label bound held in all 25.
  • With a floor (per task: Task.confidence_floor, [manager] confidence_floor for new tasks, [tasks."<key>"], jevstiller admin floor <key> 0.6, POST /jevstiller/v1/tasks/{key}/floor):
    • A local answer also counts as disagreeing when Jev would have answered with less confidence than the floor, in calibration, the shadow test and the audit. So target_agreement covers the caller's unsure decision.
    • Local answers report at least the floor, so your check keeps them.
    • Requests Jev would be unsure about go to Jev, and its own confidence comes back.
  • The cost is coverage. On the five benchmark tasks it was 4–12 points at a floor of 0.6 and 10–19 at 0.8 (experiments/confidence_floor.py, docs/benchmarks.md).
  • Changing a task's floor trains a student calibrated for it at once. Production keeps serving as it was calibrated until the new one passes shadow; the status report says so.
  • Jevstiller.evaluate(..., teacher_confidences=...) counts the same way offline.
  • Tasks without a floor behave as before.

• A local answer's Result.confidence in the Python API is now Jev's definition (the peakedness of the distribution), as the proxy already returned. It used to be the student's top probability, so the same number meant different things depending on who answered: with 2 classes, 0.8 there is 0.6 in Jev's terms. Routing is unchanged: its threshold and the stored student_confidence are still the top probability.

• The license is now Apache 2.0 (was MIT). It is just as permissive, and adds an explicit patent grant from contributors and a NOTICE file that redistributions keep. Releases up to 0.3.4 stay available under MIT.

v0.3.4

• Disk use no longer grows with every retrain. The 24-hour soak's data directory reached 53 GB, 23 GB of it old model versions: every superseded version was kept (~8–10 MB each, 645 in the busiest task). Now only the newest keep_versions (default 3) versions of each finished state (superseded, rejected, rolled back) keep their files. Production, the shadow and candidates always stay. Older ones stay listed as deleted, and rollback goes back at most that far. Versions an earlier release kept are deleted on each task's first maintenance pass after the upgrade.

• A retrain waits for 2,000 new Jev answers, not 2,000 new requests (min_new_samples). Requests the student answered counted too, so at 97% answered locally a busy task retrained every ~2 minutes on ~60 new answers, each time storing a new version. Drift and teacher changes still retrain at once. The count is an index seek, so it costs the number of new answers.

• Stored requests are bounded too. Each maintenance pass deletes the rows nothing reads any more:

  • With a Jev answer: all but the newest max_train_samples (50,000) training and max_calib_samples (20,000) calibration rows per teacher lineage. Training and calibration never read more than those.
  • Answered by the student alone: all but the newest keep_local_rows (new setting, 10,000).
  • Kept anyway: a shadow's rows, until it's judged.

  Status totals (requests, who answered, teacher calls and cost) still count deleted rows. A new store hands the space back as it goes. A store from an earlier release is rebuilt once its backlog is gone. A busy task's store now levels off at ~300 MB. On the 24-hour soak's data, everything together goes from 53 GB to 4.3 GB. After upgrading, a busy task's backlog is deleted over its first maintenance passes, 20,000 rows at a time. Expect disk I/O for a while: the soak's 6.8 million rows took 18 minutes.

• Docs: calib_fraction is a draw per request (since 0.3.0), not per text; the disk sizing in docs/deploy.md was low (~2 KB per request, no version growth).

• rare_classes defaults to "defer". A task whose teacher never uses one of its labels used to wait for that label forever before training its first student (the benchmark's CLINC150 run: Jev never answered reminder_update in 12,000 messages). Now the loop trains once the other thresholds are met and keeps forwarding any request the student would label with a class that is still below min_samples_per_class. Set rare_classes = "wait" for the old behaviour.

• Reproduce the headline result without an API key. bash experiments/reproduce.sh replays Banking77 against Jev's recorded answers, which now ship with the repository (experiments/cache/banking77.jsonl.gz), and prints the numbers next to the published ones. experiments/run.py has a --teacher cached mode for it, and experiments/record_answers.py records a complete cache for a dataset.

• Replays are reproducible. The per-request train/calibration split is now seeded from Config.seed and the task version instead of the store's file path, so the same replay gives the same result on any machine. Production behaviour is unchanged: the split is still a uniform draw per request.

• Status.report(events=False) leaves out the loop events; floats in event lines are rounded to four decimals.

• The 24-hour soak is written up (docs/benchmarks.md §Soak): 3.6 million requests, 0.012% errors, memory bounded (peak 654 MB, 308 MB at the end), silent drift at hour 12 recovered without intervention. The README's memory claim now says "bounded" rather than "flat".

• A benchmark, runnable by anyone, with results (docs/benchmarks.md §Benchmark): on five public tasks the loop reaches 22–80% coverage at 98.7–99.7% agreement with Jev and matches Jev's accuracy on every task; over 100 random splits the calibrated threshold broke the 2% budget once while the point-estimate rule broke it on 6–12 of 20 splits per task. bash experiments/bench.sh records Jev's answers for five public tasks (Banking77, CLINC150, a 20k AG News sample, TweetEval sentiment and offensive), replays each through the loop, runs experiments/baselines.py (four threshold rules on identical splits: soft or hard labels × Clopper-Pearson bound or point estimate) and rebuilds the tables at the top of docs/benchmarks.md with experiments/bench.py. experiments/target_curve.py adds the coverage-vs-target table (what admin target buys on each task). run.py --rare-classes defer trains when Jev never uses one of a task's labels (CLINC150 needed it). AG News is now read from CSV (the datasets import shadowed the local module).

• examples/race.py: 200 decisions one at a time, straight to Jev and through the proxy, recorded separately and replayed on one clock: 60.2 s vs 24.7 s (docs/benchmarks.md, docs/media/race.gif). The website now links images in docs as the files themselves.

• The pitch no longer leans on Jev's rate limit. On 2026-09-26 one key sustained 190 requests/s to Jev for a minute with no 429 (published limit: 1,200/min); TypeSafe says limits adjust dynamically. The README's "Why" now argues from latency (~300 ms per answer at every load, ~15 ms locally), sequential decisions, egress and vendor dependence; the measurements are in docs/benchmarks.md.

• The README opens with the proxy in front of live Jev. examples/proxy_demo.py is a service that classifies 5,000 real customer messages with the unmodified SDK; recorded through docker run ... jevstiller it shows answers moving from Jev (~350 ms) to the local model (~40 ms at 8 concurrent callers) after about 4,000 requests, with no errors (docs/media/proxy.gif).

• examples/quickstart_synthetic.py starts from a fresh temporary directory on every run.

• Repository: GitHub Discussions for questions and results, a pull request template, links from the issue chooser.

Jev answers for the benchmark tasks (recorded 2026-09-26)

Jev's answers (jev-1.13.0), with full probability distributions, for every message of the benchmark tasks: CLINC150 (12,000-row sample), AG News (20,000-row sample), TweetEval sentiment and offensive. Banking77's ship in the repository. bash experiments/bench.sh --no-record downloads these and reproduces the tables in docs/benchmarks.md without an API key. Not a software release: install Jevstiller from PyPI or the container image.

v0.3.3

• jevstiller admin works next to the server without flags. When no token or URL is given, it uses the server's own settings: the config file (JEVSTILLER_CONFIG), JEVSTILLER_ADMIN_TOKEN_FILE and the port. So docker exec jevstiller jevstiller admin tasks (or docker compose exec, or kubectl exec) just works.
• The Kubernetes manifest passes its config file as JEVSTILLER_CONFIG instead of --config, so kubectl exec ... jevstiller admin finds the same settings.
• Website: the getting-started page has install tabs (Docker, Compose, Kubernetes, pip), a "check it works" step with the expected output, and a screenshot of the status page.

v0.3.2

• One install command. pip install jevstiller now includes everything most people need: the proxy, the default encoder (ONNX Runtime, CPU) and the Jev adapter. On a GPU machine, pip install "jevstiller[gpu]" adds PyTorch, which is used automatically when CUDA is available. ONNX Runtime is required only on platforms it ships for (x86-64 and ARM64); elsewhere jevstiller serve says what to do.
  • The [server], [onnx], [jev], [torch] and [all] extras still work (the first three are now empty, the others mean [gpu]).
  • [gpu] used to mean ONNX Runtime for CUDA, which can't be installed next to the CPU ONNX Runtime; it now means PyTorch.
  • The Docker image's EXTRAS build argument defaults to nothing; a GPU image uses EXTRAS=gpu.

v0.3.1

• pip install jevstiller now includes the proxy. Its dependencies (Starlette, uvicorn, httpx, anyio, h11) are part of the plain install. Before, jevstiller serve failed with No module named 'uvicorn' unless you had installed jevstiller[server]. The server extra still exists (empty), so existing install commands keep working.
• For the default encoder, install jevstiller[onnx]. Without it, jevstiller serve and load_encoder now say so (pip install "jevstiller[onnx]", or use --encoder hash) instead of failing with a traceback.

v0.3.0

A status page, a defined public Python API, and the fixes from a third security audit, including two in the checks behind the agreement guarantee.

Upgrading from 0.2.0:

• Settings: a blank JEVSTILLER_* value, an empty JEVSTILLER_CONFIG or --config "", and a list of only commas now stop startup. Use none to clear a value.

• New-task quota: each API key may create at most max_new_tasks_per_key (100) new tasks per admit_window_s (24 h). Raise it, or set none, if one service legitimately starts more.

• Readiness: /readyz no longer includes data_dir_writable; watch jevstiller_data_dir_writable instead.

• Python API:

  • code that imported internal modules must import from jevstiller (or use the _-prefixed module);
  • Jevstiller's options after data_dir are keyword-only;
  • train_pool is exported from jevstiller.

• Data directories from 0.2.0 work as they are. Existing rows keep their calibration split; new rows are split per request.

• Security audit run 3 (2026-09-25; docs/security.md): fixes with regression tests in tests/test_audit_run3.py.

  • The audit now scores what was served. The drift monitor, the status report and the status page measured agreement by re-scoring the audit window with the current production student, which had since been trained on most of those rows. That overstated agreement (about 1.6x less disagreement with default settings) and could miss a broken budget. The audit now uses each audit row's recorded answer from the version in production.
  • Calibration is split per request, not per text. A text that is a large share of a task's traffic could dominate the calibration set and break the bound on the real mix. Existing rows keep their split; new rows are split by an independent draw.
  • Stored request text is cut to 32,768 characters, like the encoder's input. It used to be stored whole (up to the body limit), once per task in the request, including locally answered ones. The text hash still covers the whole state.
  • A per-key quota on new tasks: max_new_tasks_per_key (default 100 per admit_window_s), so one caller can't fill a shared tenant's task cap. Beyond it, new questions pass through (caller_task_limit). TaskManager has max_new_tasks_per_caller and route(..., caller=...).
  • Blank settings stop startup: an empty JEVSTILLER_CONFIG or --config "" (it used to skip the file, access controls and all), any empty JEVSTILLER_* value, and a list of only commas. none still clears a value.
  • Malformed requests get a 4xx, not a 500 and a traceback: a non-ASCII Basic credential on the status page is a 401; a URL httpx can't build is a 400. Paths with a backslash, a leading //, or a dot, slash or backslash still percent-encoded after decoding (%2e, %2f, %5c) are rejected (400).
  • A full data dir no longer fails /readyz. Serving goes on when records can't be written, but failing readiness took the only replica out of the Service. The new gauge jevstiller_data_dir_writable reports it.
  • Release builds are hash-pinned end to end. The build tools come from build-requirements.txt (with hashes) and the project is built without build isolation. In the image, every package is installed with --require-hashes. release.yml builds and uploads in one job with nothing else in it, tests in another, and publishes only if the files' hashes match what the build produced. The 0.2.0 notes said "hash-checked"; that covered the dependencies but not the build tools.
  • Settings errors never print a secret: a tenants map written the wrong way round printed the raw key; a token given where its file's path belongs printed the token.
  • docker build --build-arg PRELOAD_ENCODER= works: /models is writable by the runtime user when nothing is preloaded.

• A status page (P5.7) at GET /jevstiller/status. Open it in a browser and log in with the admin token as the password (any user name). It shows the share answered locally, each task's state, local share and agreement with Jev against its target, and recent events. It is read-only and cached for 5 s, and it never loads a task. It is HTML-escaped with no scripts (the CSP forbids them), and it is off without an admin token. The JSON admin API still takes only the bearer token.

• A defined public API (P7.7). It is what jevstiller, jevstiller.server, jevstiller.encoders, jevstiller.teachers and jevstiller.teachers.jev export in __all__, recorded in tests/public_api.txt and checked by a test. Everything else is internal (docs/compatibility.md). For code that imported internals:

  • Internal modules have a leading underscore: jevstiller.core → jevstiller._core, and likewise manager, store, task, training, scheduler, registry, calibrate, ood, student, settings, admin, backup, metrics, cli, the encoder and teacher implementations, and the proxy's internals (jevstiller.server keeps create_app, ProxySettings, KeyRegistry and load_salt).
  • Import from jevstiller instead: train_pool (was jevstiller.training), State, Routed, Routing and TaskInfo.
  • Jevstiller(task, teacher, data_dir, *, encoder=..., config=..., train_executor=..., hash_key=...): the options are keyword-only.
  • The engine's methods for the task manager are private: carry, restore, training_priority, busy, footprint_bytes.
  • The jevstiller command and the proxy's HTTP behaviour are unchanged.

v0.2.0

The drop-in Jev proxy. Point TYPESAFE_BASE_URL at jevstiller serve, keep your services' own Jev keys, and the proxy learns each repeated choice question from Jev's answers, then answers it locally within the agreement budget you set, with a permanent audit to Jev and automatic fallback. First release on PyPI (pip install "jevstiller[server,onnx]") and as a container image (ghcr.io/tomerglick57/jevstiller).

Upgrading from 0.1.0: existing task directories and sample stores are migrated on open, and stored tasks are re-keyed automatically. RoutingPolicy.disagreement_ub now means the per-request bound (versions trained before keep their old numbers). Teacher failures are per item (TeacherError, or errors="return").

• Packaging: published to PyPI by the release workflow (trusted publishing; the built wheel must pass the test suite first), with the README's links pointing to GitHub. The container image is published to ghcr.io/tomerglick57/jevstiller for linux/amd64 and linux/arm64, with build provenance and an SBOM. The image installs dependencies at the versions in uv.lock, hash-checked. deploy/smoke_test.py checks any image end to end (non-root, read-only, no capabilities, offline start, forwarding, admin API, clean stop), and CI runs it on every image build. Dependabot keeps the pinned actions current.
• Security audit run 2 (2026-09-25; docs/security.md): 15 findings, all fixed, with regression tests in tests/test_audit_run2.py. None exposed keys, other tenants' data or the admin API. Behaviour changes:
  • Settings are stricter. Values of the wrong type stop startup (store_text = "false" used to mean true). So do an empty secret, secret file or access-control variable (an empty JEVSTILLER_ACCESS_TOKEN used to switch the token off), text_retention_days / idle_ttl_days ≤ 0, a non-finite key_ttl_s, and trust_forwarded_for entries with host bits (10.0.0.5/24; uvicorn ignored them). store_text = false under [engine] now works (either one saying false wins). JEVSTILLER_<LIST>=none clears a list from the config file. The startup log names the access controls in effect and whether request text is stored. Credentials in upstream are redacted in logs and jevstiller config.
  • Encoder backpressure can't latch. One very large request (or one stalled encode) used to switch local answers off for every tenant until restart. The encoder now sees at most 32,768 characters of a text (model encoders read only 256 tokens anyway), and an idle encoder is never "overloaded". Embeddings of texts longer than that change once.
  • Admission counts only questions Jev answered. A caller could create tasks with questions Jev rejects. The request whose answer admits a task is recorded as its first row.
  • Forwarding never fails with a 500 or leaks a routed task: header bytes are forwarded raw (a UTF-8 User-Agent used to give a 500 and pin the task in memory until restart); any forwarding failure is a 502; retry-after is capped at 1 h.
  • No more crash when a store closes under text retention (SIGSEGV): retention and admin calls hold the task's engine; admin calls don't count as use of the task.
  • Paths with control characters or an encoded ?/# are rejected (400). They could escape a path prefix in upstream.
  • /readyz no longer flaps under concurrent probes.
  • Training workers start with forkserver on every Python version (they were forked from the server before 3.14).
  • jevstiller backup / restore write owner-only files; jevstiller admin URL-escapes names and requires the key argument; class names are escaped in admin status.
  • Also: no duplicate server/date headers; ci.yml has read-only permissions; --build-arg PRELOAD_ENCODER= works; uvicorn>=0.31.
• The agreement guarantee is now sound (found by a prior-art review, 2026-09-25). The threshold rule used to bound coverage × UB(disagreement | answered), treating the estimated coverage as exact. It kept the widest of ~200 thresholds that each passed at the full δ, which has no δ-level guarantee. And it chose the OOD cutoff on the same calibration rows. Now:
  • the rate of answered and disagreeing requests over all calibration rows is bounded directly (exact Clopper–Pearson);
  • thresholds from a fixed grid are tested strictest-first, stopping at the first failure (fixed-sequence testing, as in Learn Then Test);
  • the OOD cutoff comes from leave-one-out scores of the training reference;
  • shadow judging uses the same loss.
  • tests/test_calibrate.py simulates 300 calibration sets and checks that the chosen threshold breaks the budget in at most ~δ of them.
  • RoutingPolicy.disagreement_ub now means that per-request bound. Versions trained before this keep their old numbers.
• Connection reuse: jevstiller serve keeps idle connections for 75 s (it was uvicorn's 5 s, the same as httpx's reuse window, and requests in flight were dropped: 37 in 1.15M in the soak). A forwarded request whose pooled connection Jev had just closed is retried once (7 in 1.15M were 502s).
• README: How it compares, covering stuntd, Distil Labs, routers, caches, open Jev-compatible models and the research lineage.
• Phase 6 testing found and fixed:
  • Silent drift never recovered. When Jev changed its answers without changing its model name, a task fell back and then retrained forever on mixed old and new answers; every candidate failed shadow. A confirmed drift (fallback) now restarts the training data: training, calibration, shadow and audit read only rows after the event's since_id. A replayed silent drift is back to 81% local answers within 2,000 requests and to its old ~96% within ~20,000. The fallback survives a restart; mode auto ends it by hand (fallback_cleared event). After a teacher change or a drift, the next student waits for the full readiness thresholds on the new data.
  • Memory grew without bound when tasks load and unload often (more active tasks than max_loaded): 555 MB after 4 minutes and accelerating. A reference cycle (engine → training executor → the engine's priority method) kept every unloaded engine and its arrays until a full garbage collection. The scheduler and training callbacks now hold engines weakly; an unloaded engine is freed at once (tested with the collector off). In the final 40-minute soak (~18 reloads a second and a drift) memory ended at 182 MB.
  • max_loaded didn't hold under a stream of loads. Only the janitor enforced it, one unload at a time, and it skipped every engine with a maintenance pass merely requested, which under steady traffic is nearly all of them. benchmarks/manager.py ended up with 505–666 tasks loaded against max_loaded = 50 (2.7 GB). Now a load that pushes past the cap unloads the least recently used idle engine itself, and only a running pass or a pending training job keeps an engine loaded.
  • glibc heap fragmentation under task churn: every load starts threads and SQLite connections, spread over up to 8 arenas per core. Resident memory grew ~47 KB per load/unload with Python's own allocations flat. The manager now limits glibc to 2 arenas (unless MALLOC_ARENA_MAX is set): 7,365 load/unload cycles held 234 → 300 MB, with serving latency unchanged.
  • Progress was lost on every reload: a shadow's sample count restarted (so an often-unloaded task never got its shadow judged), the "rows since last training" counter restarted (a retrain on every reload once a task had min_new_samples rows), and the drift-check counter restarted. Versions now record trained_to_id and shadow_from_id, and the manager carries the in-memory counters across unloads.
  • Training workers outlived a kill -9 of the server, with the forkserver and resource tracker: they never see their job queue close. Each worker now exits within a second of the server's death.
  • A task without a student for its current data (new, or after a teacher change or drift) whose candidate failed retrained every 100 rows. It now waits for 25% more data (at most min_new_samples).
  • Beyond the encoder's capacity, latency grew without bound (the load test: 15 s p99 at 256 callers with bge-small on CPU, and throughput halved). Requests that would wait longer than max_encoder_wait_ms (200) for the shared encoder are now forwarded to Jev unrouted (overloaded); jevstiller_encoder_wait_seconds shows the backlog. Also, requests that are only forwarded (questions not admitted as tasks) are no longer encoded at all.
  • Forwarding collapsed beyond ~100 concurrent requests (~75 req/s at 256 callers). The cause was the single httpx client to Jev: its pool's cost grows with connections × waiting requests. Upstream connections are now split over clients of 16 (UPSTREAM_POOL), with each request going to the least busy one: 614 req/s at 256 callers. The same limit in the load generator is why benchmarks/load.py spreads its connections over processes.
  • A failure while routing a request from a not-yet-verified key could raise after Jev had already answered; it now forwards Jev's answer unrecorded.
  • At the default info level, httpx logged a line per forwarded request next to the access log; it now logs warnings only.
  • A fit read a task's whole history. Training time and the training worker's memory grew without bound with a busy task's age (a task with 100k teacher answers a day would hold ~4.6 GB after a month). Fits now use the most recent max_train_samples (50,000) and max_calib_samples (20,000) rows.
  • A graceful shutdown during a training job left an empty staging directory (removed at the next start anyway); it is now removed.
  • A full disk logged a stack trace per failed batch. Now one line per store per minute, and the new counter jevstiller_store_dropped_records_total. Serving continues either way.
• Tests: `tests/test_chaos...