You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The incumbent tests.lock enumerated every case (16,427 names + tags) and the runner required exact per-target agreement: missing case, extra case, and tag drift were each hard errors (7110a99:conformance/runner/src/main.rs:222-247). The component-test lock stores, for the ~17,960 vector/contract-derived cases, only [[generated]] row prefixes + tags — no case list, no count (guest-ct/tests.lock; format at component-test lockfile.rs:36-43). Verified consequences in component-test source at the pin:
check_coverage requires exact [[case]] entries (probes/declines, ~59 of ~18k) but for generated leaves checks only grammar and at-most-once — no lower bound; a row reporting zero leaves passes (lockfile.rs:164-212).
The aggregate's per-target coverage is exactly check_coverage (aggregate.rs:228-230); no cross-target leaf-set agreement exists, so a uniform drop (same suite artifact everywhere) is invisible to aggregation.
lock --check compares prefixes + tags only (component-test-cli main.rs:98-105).
Residual pins, per row class:
Incumbent rows: the native census test expands plan::ROWS and requires exact id+tag equality with the frozen fixture (census_test.rs), so a translate-level regression fails it — but only via the JSON path; a regression confined to the rkyv emission (build.rs/corpus.rs row splitting) is unobserved by it.
Post-cutover rows (ecdsa-p256-sha512/wycheproof, ecdsa-p384-sha512/wycheproof from conformance: vendor the SHA-512 ECDSA P1363 files; execute the cross-hash verify paths #299): deliberately outside plan::ROWS and the fixture. Their populations are pinned by exactly one artifact: the committed matrix row label ("816 cases") diffed by matrix-check — which is CI-only (not part of conformance-ct::all locally), count-granular (a same-count case substitution passes every gate; the old lock diff listed each changed case name), and refreshed wholesale by matrix-update (cp).
conformance/README.md's claims are currently stronger than the mechanism: "Growing the suites therefore never silently sheds coverage" and "the inventory is the binding" hold at prefix granularity for ~99.6% of cases.
Fix shapes (either restores a real lower bound):
per-row case counts or a leaf-set digest in the lockfile format (upstream component-test change), or
a census-test row per new prefix (a second, growing fixture — the frozen incumbent fixture stays frozen), plus adding matrix-check to local all.
Related: #83 records that the old lock pinned inventory but not assertions; this issue records that the new lock pins neither, per-case, for generated rows. The S5 ratification issue (filed separately) depends on this one.
Found by adversarial review of the #274 migration; mechanics independently confirmed by two reviewers against component-test at c90b2f0.
The incumbent tests.lock enumerated every case (16,427 names + tags) and the runner required exact per-target agreement: missing case, extra case, and tag drift were each hard errors (7110a99:conformance/runner/src/main.rs:222-247). The component-test lock stores, for the ~17,960 vector/contract-derived cases, only
[[generated]]row prefixes + tags — no case list, no count (guest-ct/tests.lock; format at component-test lockfile.rs:36-43). Verified consequences in component-test source at the pin:check_coveragerequires exact[[case]]entries (probes/declines, ~59 of ~18k) but for generated leaves checks only grammar and at-most-once — no lower bound; a row reporting zero leaves passes (lockfile.rs:164-212).check_coverage(aggregate.rs:228-230); no cross-target leaf-set agreement exists, so a uniform drop (same suite artifact everywhere) is invisible to aggregation.lock --checkcompares prefixes + tags only (component-test-cli main.rs:98-105).Residual pins, per row class:
plan::ROWSand requires exact id+tag equality with the frozen fixture (census_test.rs), so a translate-level regression fails it — but only via the JSON path; a regression confined to the rkyv emission (build.rs/corpus.rs row splitting) is unobserved by it.ecdsa-p256-sha512/wycheproof,ecdsa-p384-sha512/wycheprooffrom conformance: vendor the SHA-512 ECDSA P1363 files; execute the cross-hash verify paths #299): deliberately outsideplan::ROWSand the fixture. Their populations are pinned by exactly one artifact: the committed matrix row label ("816 cases") diffed bymatrix-check— which is CI-only (not part ofconformance-ct::alllocally), count-granular (a same-count case substitution passes every gate; the old lock diff listed each changed case name), and refreshed wholesale bymatrix-update(cp).conformance/README.md's claims are currently stronger than the mechanism: "Growing the suites therefore never silently sheds coverage" and "the inventory is the binding" hold at prefix granularity for ~99.6% of cases.
Fix shapes (either restores a real lower bound):
matrix-checkto localall.Related: #83 records that the old lock pinned inventory but not assertions; this issue records that the new lock pins neither, per-case, for generated rows. The S5 ratification issue (filed separately) depends on this one.
Found by adversarial review of the #274 migration; mechanics independently confirmed by two reviewers against component-test at c90b2f0.