Roster expansion Step 1: ingest 13 real supersession corpora (no gold labels)#20
Open
tcballard wants to merge 1 commit into
Open
Roster expansion Step 1: ingest 13 real supersession corpora (no gold labels)#20tcballard wants to merge 1 commit into
tcballard wants to merge 1 commit into
Conversation
…sion (Step 1) Stages the corpora for the graded-overlap roster expansion in a new scenarios_candidates/ directory: 10 PEP + 3 RFC real supersession edges, each with a reproducible corpus + provenance.json and NO scenario.json (gold labels are authored blind later, per CONTRIBUTING rule 1). Edges were selected empirically, not guessed: enumerated from the pinned pool provenance and cross-checked against the existing roster to find valid, unused supersession edges, each derived from the artifacts' own Replaces/Superseded-By (PEP) or Obsoletes (RFC) headers and verified to reproduce from the pin. Topics span the overlap spectrum (typing/class creation = high; packaging metadata/hashlib = medium; governance/TLS API/TCP/IPv6 = low) so a blind author can write graded-overlap tasks. Fully inert: scenarios_candidates/ has no scenario.json, so neither load_scenarios nor test_real_roster.py::PINNED_ROSTER sees it — the frozen 49-scenario roster is untouched (verified). Promotion into scenarios_real/ remains a deliberate pre-registration event (blind gold label + roster re-pin + a new amendment). Refreshes docs/real-scenario-candidates.md to the confirmed set and adds scenarios_candidates/README.md.
This was referenced Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 1 of the reportable real-roster expansion: stage real, pinned corpora so a blind author can write graded-overlap scenarios on top. Corpora + provenance only — no gold labels, nothing enters the frozen roster.
What's here
13 real supersession edges (10 PEP + 3 RFC) ingested into a new
scenarios_candidates/directory, each with a reproduciblecorpus/*.md+provenance.jsonand noscenario.json.peps_class_creation_supersessionpeps_typeis_narrowing_supersessionpeps_single_dispatch_supersessionpeps_metadata_v12_supersessionpeps_package_layout_supersessionpeps_hashlib_api_supersessionpeps_packaging_governance_supersessionpeps_windows_installer_supersessionpeps_tls_api_supersessionpeps_pypi_mirror_supersessionrfc_utf8_supersessionrfc_tcp_supersessionrfc_ipv6_supersessionHow the edges were chosen
Empirically, not guessed. I enumerated every real supersession edge at the pin from the committed pool provenance, cross-referenced against the existing roster to find valid unused edges, and validated each by running the ingest tool and confirming it derived the edge from the artifacts' own
Replaces/Superseded-By(PEP) orObsoletes(RFC) headers. All 13 passpython -m ingest.<tool> verify(reproduce byte-for-byte from the pin). Topics span the overlap spectrum so a blind author has high/medium/low material.Safety — the frozen roster is untouched
tests/test_real_roster.py::_on_disk()only counts directories containing ascenario.json, soscenarios_candidates/is invisible to both the roster pin (still exactly 49) andload_scenarios. These corpora are inert until promoted. Full suite green (440 passed, 6 skipped).Next steps (not in this PR)
scenario.json(task + gold label) blind, perspec/external-scenario-request-graded-overlap.md; must pass the offline gate and match its declaredlexical_overlapband.git mveach intoscenarios_real/, extendPINNED_ROSTER+ per-domain counts, and land a new analysis-plan amendment re-registering the enlarged roster (a deliberate pre-registration event).Scope covered here is supersession edges only (mechanically verifiable from headers); prohibition/conflicting real scenarios are a follow-on since they have no machine-readable edge. See
scenarios_candidates/README.mdanddocs/real-scenario-candidates.md.Generated by Claude Code