You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
sergio velderrain edited this page Nov 11, 2026
·
2 revisions
TEST-126
Stimulus
Feed a calibration scorer fixed Windows 11 workload seeds and monotonic event/response/action timestamps for on-demand, periodic, and continuous modes. Include first-token, final-token, oracle-action, state-grounding, and task-success outcomes with known sample counts and uncertainty. Attempt to mix modes, use a missing model digest, omit capture configuration, tune thresholds from qualification data, or claim qualification before a reviewed threshold version is frozen.
Required observation
Offline scorer cases test_calibration_summarizes_each_mode_separately_and_never_qualifies, test_calibration_rejects_mixed_seeds_and_inconsistent_configuration, test_calibration_rejects_malformed_records, and test_calibration_single_sample_reports_unestimated_uncertainty passed through PR #254, merge 61d6e6dd3ff435d6d30c5edc370aea4d61f2236f. The scorer reports per-mode p50/p95, grounding/task-success measures, sample counts, and uncertainty, and rejects malformed/mixed records. No calibration dataset or numeric threshold exists; this is not live baseline or qualification evidence.