epic: finalize candidate generation and First Evaluation

[aryamantodkar]

Summary

Merge the completed candidate-generation latency epic into main.

This epic contains the Rust batch realization optimization, exact equivalence coverage, the retained First Evaluation comparison, optional reproducibility runners, and fail-closed benchmark provenance checks.

Runtime changes

• Batch all span/kind realizations through one Python-to-Rust call.
• Parse kinds once, deduplicate replacements in Rust, release the GIL, and parallelize independent spans with Rayon.
• Preserve exact candidate tuple content and ordering.
• Add native build metadata through _rust.build_info().

Benchmark safeguards

The comparison runner refuses to start if:

• the dataset path or SHA-256 differs from the frozen manifest;
• the checkpoint SHA-256 differs from production metadata;
• production and benchmark checkpoint digests disagree;
• the frozen model name or revision differs;
• the Rust extension is not a release build;
• debug assertions are enabled.

It records the verified dataset, checkpoint, model, and imported native extension hashes in run.json.

First Evaluation

Backend	Overall semantic accuracy	Voice-agent accuracy	Mean latency
Premove ITN	89.70%	99.50% (398/400)	56.49 ms
Thutmose	59.39%	67.00% (268/400)	15.98 ms
text-processing-rs	55.79%	68.25% (273/400)	0.14 ms

The retained report includes overall metrics, voice-agent-only domain metrics, entity-category metrics, warm-up timings, per-record latency, and Premove timing components.

The invalid development-build result set was removed from the repository. Only eval/voice_agent_itn/results/first-evaluation/ remains.

Validation

• uv run ruff check .
• cargo fmt --manifest-path rust/Cargo.toml --check
• cargo test --manifest-path rust/Cargo.toml — 60 passed
• uv run pytest — 116 passed
• uv build
• 1,500/1,500 candidate graphs exactly equal to the pre-batch implementation
• artifact preflight tests cover changed dataset, checkpoint, model revision, and debug extension failures

Documentation

• First Evaluation report
• Benchmark reproduction guide
• VoiceAgent benchmark specification
• Model provenance

This PR is the epic promotion from epic/candidate-generation-latency to main.