Skip to content

TEST 126

sergio velderrain edited this page Nov 11, 2026 · 2 revisions

TEST-126

Stimulus

Feed a calibration scorer fixed Windows 11 workload seeds and monotonic event/response/action timestamps for on-demand, periodic, and continuous modes. Include first-token, final-token, oracle-action, state-grounding, and task-success outcomes with known sample counts and uncertainty. Attempt to mix modes, use a missing model digest, omit capture configuration, tune thresholds from qualification data, or claim qualification before a reviewed threshold version is frozen.

Required observation

Offline scorer cases test_calibration_summarizes_each_mode_separately_and_never_qualifies, test_calibration_rejects_mixed_seeds_and_inconsistent_configuration, test_calibration_rejects_malformed_records, and test_calibration_single_sample_reports_unestimated_uncertainty passed through PR #254, merge 61d6e6dd3ff435d6d30c5edc370aea4d61f2236f. The scorer reports per-mode p50/p95, grounding/task-success measures, sample counts, and uncertainty, and rejects malformed/mixed records. No calibration dataset or numeric threshold exists; this is not live baseline or qualification evidence.

Clone this wiki locally