Repository navigation
Discussion: FID-106 — Can Independent Reviewers Reproduce a Specific AI Safety Claim? #232
alexchaomander
started this conversation in
Research Calls
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
What is the smallest public evidence package that lets someone outside the original research team reproduce a bounded result, identify where the claim goes beyond that result, or document why reproduction is blocked?
This seed call welcomes questions, reading notes, and offers to draft or review a bounded first contribution.
Starter project
Start with the pinned DSEWiki release. Use a fresh checkout at
87bb9ab6454d3a161d33fc04d101d796003b63bband Python 3.11 or later. Afterobtaining the public release, the verification is offline and uses the
standard library. Release and instructions ·
Published analysis
Run the release check from the root of the checkout.
Inspect
investigations/agent-incidents/corpus-review/results/study/analysis.json.In
paired_summary, select the row withselection = corrected_primaryand
budget_min = all. Comparecoverage_increasedandcoverage_increased_with_retained_problemwith the released counts of 61and 44, respectively.
Done means a short reproduction record includes the revision, Python
version, command, observed counts or exact blocker, and the labels and
mappings held fixed. The release check recomputes an aggregate from existing
structured assessments. A separate contributor can challenge one underlying
judgment using the release contribution guide.
Comment with the part you would like to take on, your rough availability, and any review or support you need. The contributor guide explains how to agree a first output. Use the private introduction route for contact details or confidential context.
All reactions