Repository navigation
Unittest receipt comparison: exact stderr outside the final timing field #1916
Arham-Tahir64
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
DONE — PR #1926, head
387501825842caa3507e34f110b5f9570dc04046. Claim released.The integrated PR921 benchmark can normalize away substantive diagnostic changes: its unanchored timing regex erases
in 1.000sversusin 2.000sin failure text, and UTF-8 replacement decoding collapses distinctff/febytes. Both collisions reproduce with real unittest captures; bounded fault injection makes the frozen benchmark report PASS for changed sequential/concurrent stderr.The source-bound successor package
research/unittest-receipt-comparisonretains all captured bytes except the numeric elapsed value in the final standard unittest trailer. Separate raw/summary tags and an explicit prefix length prevent marker collisions. An opt-in adapter preserves the predecessor's commands, test inventory, stdout and exit-code checks, and rejection of remaining mismatches; predecessor files stay untouched.Verified locally on Windows 11 / CPython 3.12.10: all 20 focused tests pass in both normal and optimized registered replay, with genuine timing-only acceptance, exact bytes, real captures, end-to-end faults and source-drift controls. Metadata/pins, affected-package selection and patch hygiene pass. All 42 onboarding tests pass with PYTHONUTF8=1 (the separate cp1252 issue is PR1834); workspace TEMP/TMP supplied sandbox-compatible isolated fixtures.
Reproduce:
python3.12 -B scripts/research_package.py --package research/unittest-receipt-comparison. TASK/SOURCES/RESULT/VALIDATION/VERIFY carry the precise scope, sources, witnesses and limitations. Full repository benchmarking and selected mathematical replay remain unrun; no speedup, new kappa, historical-run fault or full-suite equivalence is asserted. Hosted CI was queued at publication and is not counted as a local pass.Base main
0b1cccf86e2f5132ed286a26ef0a681a495740ea; portable-replay nonced478b42a7ca6903045649ed7bd64bc93, priority revision 3. Final fresh overlap covered 573 open PRs and 542 Discussion records, complete pagination and no warnings. Matches were this claim/package, its frozen predecessor and the unrelated data-corner benchmark. This is not proof of novelty. Related PR1852 protects child assertions; it addresses a distinct obligation.Next question: run the exact adapter on the entire pinned suite in a supported environment and inspect any custom output that conservatively fails comparison. New trailer support must retain diagnostic byte equality.
Credit Aaron Wang/navinor PR921 (reviewed submitted head
c1ee2fbfdc5700a1ba0251e7727ee8ae0cb4782f) and Douglas Colkitt's batch932 acceptance hardening. Prepared by Arham Tahir with substantial OpenAI Codex assistance; Apache-2.0. No subagents or independent human review.All reactions