Repository navigation
Discussion: FID-103 — When Does AI-Assisted Review Miss Shared Blind Spots? #229
alexchaomander
started this conversation in
Research Calls
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
When an AI system and its reviewers rely on the same mistaken premise, which review procedure helps an accountable reviewer catch the error before accepting a consequential answer or recommendation?
This seed call welcomes questions, reading notes, and offers to draft or review a bounded first contribution.
Starter project
Start with the systematic human error questions in the UK AISI agenda and
the comparison between debate and collaborative disagreement resolution.
UK AISI alignment agenda ·
Collaborative Disagreement Resolution
Write one adjudicable source-conflict case. In a fictional institution,
two reviews approve an action because both cite an older policy. A newer
version changes who may approve the action. Write the recommendation, both
policy excerpts, their effective dates, and an adjudication note. Include a
matched control in which the older policy actually governs and a version in
which the decisive record is missing. An existing public case with equivalent
evidence is also suitable.
Done means a second reviewer can use the cited clause and date to decide
whether to accept, reject, or defer each variant. Specify what direct review
and structured review see under the same evidence access. Then specify what
changes when either gets the independently assembled evidence. A domain expert
can check authority and timing, and a methods reviewer can check for unintended
hints or ambiguity.
Comment with the part you would like to take on, your rough availability, and any review or support you need. The contributor guide explains how to agree a first output. Use the private introduction route for contact details or confidential context.
All reactions