evalhub 0.4.0

Close the evalhub.eval/1.0 window: a 1.0 body is 422 runs_moved, GET /schemas/eval is gone (0.4.0) (#54)

0.4.0: the evalhub.eval/1.0 compatibility window is closed

Closes #45 and #51. Releases 0.2.0 and 0.3.0 accepted an evalhub.eval/1.0 body (header and runs[] in one document) on POST /evals/{ns}/{name} and converted it at ingest into a 2.0 header and its runs, answering with a Deprecation header and converted_from / converted_runs in the envelope. The schema crate's rule is that the previous major is accepted for one release and then refused; the README and the crate docs said 0.3.0 would refuse it, and 0.3.0 shipped with the window open (#51). 0.4.0 closes it.

Behaviour

• POST /evals/{ns}/{name} with schema: evalhub.eval/1.0 answers 422 with exactly one error, runs_moved at /runs, with or without runs and whatever else is wrong with the body. It is the error a 2.0 header carrying runs already got. No Deprecation header, no converted_from, nothing is written. The hint names PUT /evals/{ns}/{name}/runs/{run_id} and POST /evals/{ns}/{name}/runs:batch.
• A 2.0 header carrying runs whose attachments[].sha256 is malformed also answers that single runs_moved (previously the sha error was appended after it).
• GET /schemas/eval (the 1.0 document) is gone: 404. GET /schemas/eval-2 and GET /schemas/run are unchanged.
• Stored data is untouched. A 0.1.x database still migrates: 0003_runs_split keeps its own reading of the 1.0 shape inside evalhub-store (unpublished), byte-for-byte the conversion evalhub-core used to expose.
• Producers still sending 1.0 bodies get 422 after this deploy; the README's 0.1.x → 0.2.0 mapping stays as the record of what to change.

SDK (why 0.4.0)

just semver-check reports major for evalhub-schema and evalhub-core; evalhub-query is unchanged.

• evalhub-schema: eval::v1 (Eval, Run, Outcome) and schema_for_eval_v1() removed; SCHEMA_NAMES is [&str; 5] (eval dropped; cargo-semver-checks does not flag the array length); RecordKind::Eval.schema_ids() is ["evalhub.eval/2.0"], schema_for("evalhub.eval/1.0") is None, all_schemas() no longer yields eval; schemas/eval.json removed; ErrorCode::RunsMoved doc covers the 1.0 declaration.
• evalhub-core: EvalSchema and declared_schema removed (the 1.0 arm of the validator). The eval module went with them: split_v1, SplitV1, UNRECORDED, V1_MIGRATION_META_KEY now live in evalhub-store for the data migration only. validate(Eval, …) on a 1.0 body returns [runs_moved].
• OpenAPI: ConvertedRunsDto, converted_from, converted_runs removed; the web client (schema.d.ts) regenerated.

Docs

Every sentence that promised the removal for 0.3.0 (README §API, §Upgrading from 0.1.x; rustdoc in schema, core, store, server) now records that 0.2.0 and 0.3.0 accepted 1.0 and 0.4.0 closed the window. web/README.md names eval-2 instead of the removed eval.

Verification

cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test -p evalhub-schema (20) / -core (56) / -query (27) / -store (104, Docker) / -server (86, Docker), cargo insta test -p evalhub-schema and -p evalhub-server (no pending snapshots), RUSTDOCFLAGS=-D warnings cargo doc for schema / core / store, just semver-check (major for schema and core before the bump; "no semver update required" for all three at 0.4.0), just package (three crates ok), corepack pnpm check / pnpm lint in web/. RED observed before each behaviour change: schema_ids() still listed 1.0; a 1.0 body validated with zero or unrelated errors; POST answered 201 with converted_from; GET /schemas/eval answered 200; a malformed sha256 produced two errors.

Fly is deployed separately after the release (fly deploy --ha=false); nothing stored changes, so no pre-deploy data check is needed. Producers still posting 1.0 bodies get 422 runs_moved from then on.

---

SDK crates: evalhub-schema evalhub-core evalhub-query
Image: ghcr.io/ynishi/evalhub:0.4.0