Parent: #456. Non-gating for the tag — this is stated in #456's own checklist and in participant-validation.md. Blocked on Cut D (#509): the case cards print the frozen label and what the verifier concluded, so neither exists until receipts do.
Why it is worth doing anyway
The corpus's real-history cases were written and reviewed by real people. On the question the corpus is actually about — should this change have been stopped? — those people are better authorities than any rater we can supply, because they had the context at the time. This gate asks them, and records what comes back.
It also does double duty: the same conversation validates a case and asks the product-signal question, which makes it the first evidence-first outreach wave rather than a separate ask.
The five pre-registered rules
Fixed in Amendment 1 so this cannot drift into after-the-fact relabeling:
- Frozen labels do not move. A material disagreement triggers a case review whose corrections land in the
beta corpus — except a material error found before the tag ships, which may force a re-freeze.
- Reviewers outrank authors. Both are asked; recorded separately.
- Two questions, recorded apart. "Is the label right?" and "would this have been useful?" have different consumers.
- Agreement-among-responders reporting. Silence is not assent.
- Naming needs consent, asked inside the outreach message. Absent a yes, the response is aggregate-only.
Contact protocol
Per real_history / rejected_or_reverted case: the reviewer whose decision resolved the PR first, then the author. Design-partner cases go through the partner contact. No bulk mail, no bot comments, at most one follow-up. A non-response is a data point, not an invitation to persist.
Done when
Parent: #456. Non-gating for the tag — this is stated in #456's own checklist and in
participant-validation.md. Blocked on Cut D (#509): the case cards print the frozen label and what the verifier concluded, so neither exists until receipts do.Why it is worth doing anyway
The corpus's real-history cases were written and reviewed by real people. On the question the corpus is actually about — should this change have been stopped? — those people are better authorities than any rater we can supply, because they had the context at the time. This gate asks them, and records what comes back.
It also does double duty: the same conversation validates a case and asks the product-signal question, which makes it the first evidence-first outreach wave rather than a separate ask.
The five pre-registered rules
Fixed in Amendment 1 so this cannot drift into after-the-fact relabeling:
betacorpus — except a material error found before the tag ships, which may force a re-freeze.Contact protocol
Per
real_history/rejected_or_revertedcase: the reviewer whose decision resolved the PR first, then the author. Design-partner cases go through the partner contact. No bulk mail, no bot comments, at most one follow-up. A non-response is a data point, not an invitation to persist.Done when
benchmark/safety-qualification/participant-validation-log.csvis committed, with the two questions recorded apart.betacorpus (Deliverable 2: the 100-case beta corpus required for 1.0 (#456) #512).