2026 05 06 fact checking tools research quality improvement

How can findings from OpenFactCheck, Loki, FActScore, gpt-oss-safeguard, and Barnum statement research be synthesised into concrete improvements to the automated review process in this research repository?

Research Question

How can the findings from research into OpenFactCheck, Loki, FActScore, gpt-oss-safeguard, and Barnum statement identification techniques be synthesised into concrete, actionable improvements to the automated review process (research-review-prompt.md) in this repository, specifically targeting factual precision, policy compliance, and output quality?

Scope

In scope:

Synthesis of findings from the five prerequisite items: capabilities, limitations, and deployment feasibility of each tool
Gap analysis between the current research-review-prompt.md review criteria and what automated claim-level verification could additionally check
Actionable recommendations for each of three intervention points: (1) the research agent prompt (research-prompt.md), (2) the review workflow prompt (research-review-prompt.md), and (3) a new automated pre-review tooling step
Assessment of which quality improvements can be made with no new tooling (prompt changes only) versus which require new tooling integration
Priority ordering of recommendations by effort-to-impact ratio
Proposed candidate rubric additions to research-review-prompt.md for: atomic claim labelling completeness, Barnum statement absence, and source URL coverage
Identification of W-XXXX backlog items to capture any tooling changes required

Out of scope:

Implementation of any tooling changes (these are scoped to separate backlog items produced as output)
Re-researching the individual tools (these are covered in the five prerequisite items and should be read as inputs to this synthesis)
General Large Language Model (LLM) evaluation methodology beyond what is directly applicable to this repository's research outputs

Constraints:

This item must not start until all five prerequisite items are completed: 2026-05-06-openfactcheck-ai-fact-checking-pipeline, 2026-05-06-loki-fact-checking-journalists-moderation, 2026-05-06-factscore-precision-scoring-atomic-claims, 2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight, and 2026-05-06-barnum-statements-ai-responses-theory-practice
Recommendations must be specific and actionable: each must identify the exact file and section to be changed, the nature of the change, and the expected quality improvement
The synthesis must distinguish between prompt-level fixes (zero infrastructure cost) and tooling fixes (require new dependencies or workflow steps)

Context

This repository runs an autonomous research loop that produces structured research items and evaluates drafted items through research-review.yml against research-review-prompt.md. [fact; source: https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md] The current review criteria are strong on source binding, epistemic labelling, section completeness, and logical coherence, but they do not make claim-level precision, policy compliance, or Barnum statements, low-specificity sentences that sound analytical without adding decision-useful detail, first-class failures. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md] This synthesis uses five prerequisite tool studies to translate those gaps into prioritised, actionable improvements to the review process. [fact; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md]

Approach

Read prerequisite findings: Summarise the key findings from each of the five prerequisite items in a structured comparison table covering: capability, accuracy, deployment feasibility (GitHub Actions compatible?), cost, and primary failure mode.
Gap analysis against current review criteria: Read research-review-prompt.md in full. Map each current review criterion to the quality dimension it addresses. Identify the quality dimensions not currently covered: factual precision at claim level, Barnum statement density, policy compliance, and source recency.
Prompt-only improvements (no new tooling): Identify the additions to research-review-prompt.md that require no new tooling, for example, explicit Barnum statement criteria, atomic claim labelling completeness checks, and a prohibition on hollow hedging phrases. Draft the proposed criterion text for each.
Tooling-assisted improvements: For improvements that would benefit from automated tooling (FActScore integration, OpenFactCheck pipeline, gpt-oss-safeguard policy check), assess: (a) integration effort, (b) false positive risk, (c) impact on review latency, and (d) whether the improvement could be approximated with a prompt-only approach first.
Priority matrix: Rank all proposed improvements on a two-axis matrix: implementation effort (low/medium/high) vs. expected quality uplift (low/medium/high). Identify the top three highest-impact, lowest-effort changes.
Backlog item drafting: For each improvement that requires a new backlog item (tooling integration, skill update, prompt rewrite), draft the item title and one-sentence scope so the output can immediately populate BACKLOG.md and/or Research/backlog/.
Research prompt improvements: Assess whether any improvements should be applied to research-prompt.md rather than research-review-prompt.md, for example, whether the research agent should be instructed to avoid Barnum statements at generation time, not just catch them at review time.

Sources

Research Skill Output

(Full output from running the research skill, retained verbatim in the completed item. §§0-5 are the investigation; §6 seeds the Findings section below.)

§0 Initialise

Question: How should this repository upgrade its automated research-review process to improve factual precision, policy compliance, and output quality without overfitting the workflow to heavyweight or brittle tooling?
Scope: Compare prompt-only fixes, review-rubric additions, and one deterministic pre-review tooling step; do not implement the changes in this item.
Constraints: Recommendations must name the exact file and section to change, distinguish zero-infrastructure changes from tooling changes, and stay compatible with the current GitHub Actions review workflow.
Output: One synthesis item with prioritized recommendations, drafted backlog candidates, and mirrored Findings.
[fact] Prior completed-item scan performed before investigation and repeated during drafting, with direct evidence taken from the five prerequisite items plus adjacent completed items on claim verification, review load, prompt engineering, and Large Language Model (LLM) as judge deployment. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]
[inference] The repository should prefer prompt and deterministic checks before heavyweight model-integrated verification because the current review workflow runs a single GitHub-hosted review job around a Command Line Interface (CLI) prompt, not a specialized retrieval or Graphics Processing Unit-backed verification service. [source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md]

§1 Question Decomposition

Current-state audit
- 1.1 What quality dimensions does research-review-prompt.md explicitly check today?
- 1.2 Which target quality dimensions in scope, atomic claim completeness, Barnum detection, policy compliance, and source recency, are not explicit first-class review checks today?
Tool synthesis
- 2.1 What does OpenFactCheck contribute that the current workflow lacks?
- 2.2 What does FActScore contribute that the current workflow lacks?
- 2.3 What does Loki contribute that the current workflow lacks?
- 2.4 What does gpt-oss-safeguard and its Llama Guard comparator contribute that the current workflow lacks?
- 2.5 What does Barnum-statement research contribute that the current workflow lacks?
Deployment-fit test
- 3.1 Which improvements can be captured as prompt-only rubric changes with zero new infrastructure?
- 3.2 Which improvements require deterministic tooling but not a new external service?
- 3.3 Which improvements require sampled or optional external-model assistance rather than mandatory inline gating?
Priority and backlog output
- 4.1 Which recommendation has the best effort-to-quality-uplift ratio?
- 4.2 Which recommendations should be drafted as backlog candidates rather than applied immediately?
- 4.3 Which upstream change belongs in research-prompt.md because it prevents defects before review?

§2 Investigation

Current review coverage

[fact] research-review-prompt.md currently orchestrates four review passes, citation-discipline, speculation-control, remove-ai-slop, and peer-reviewer, and its explicit checks focus on source binding, uncertainty labeling, prose quality, logical coherence, confidence calibration, and cross-item integration. [source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md]
[fact] The same prompt does not explicitly instruct the reviewer to test whether every evidence-bearing sentence in Findings has been decomposed into atomic claims, whether generic pseudo-analytic sentences lack a concrete anchor, or whether time-sensitive claims rely on recent sources. [source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md]
[inference] The gap matters because the current rubric can accept a sentence that is labeled, cited, and logically phrased while still bundling multiple propositions under one citation or sounding analytical without adding decision-useful specificity. [source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]

OpenFactCheck and FActScore

[fact] OpenFactCheck's primary value is modular claim verification and checker benchmarking, while its repository-fit synthesis concluded that it is better suited to optional or offline audit use than to mandatory inline gating in this lightweight repository workflow. [source: https://arxiv.org/abs/2405.05583; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md]
[fact] FActScore's primary value is atomic-fact precision scoring against a bounded knowledge source, and its repository-fit synthesis concluded that the metric is strongest as an offline diagnostic or sampled audit, not as a direct release gate for mixed-domain synthesis documents. [source: https://arxiv.org/abs/2305.14251; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]
[inference] Together these two systems justify adding atomic-claim completeness and support-critical claim sampling to the review process, but they do not justify making a full OpenFactCheck or FActScore run mandatory on every draft. [source: https://arxiv.org/abs/2405.05583; https://arxiv.org/abs/2305.14251; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]
[fact] Adjacent repository work on automated claim verification reaches the same conservative conclusion: claim verification should be bounded to support-critical claims, claims whose failure would materially change the item's recommendation or decision relevance, and unsupported matches should be downgraded rather than preserved as facts. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md]

Loki

[fact] Loki's primary value is human-centered evidence discovery through claim decomposition, check-worthiness filtering, retrieval, and claim-local verification, and its repository-fit synthesis concluded that it is better as an evidence-assistant surface than as a final automated gate. [source: https://arxiv.org/abs/2410.01794; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md]
[inference] Loki therefore supports a future sampled escalation path for disputed or support-critical claims, but it is not the first tooling investment because the current review loop needs deterministic coverage and prompt tightening before adding live-web verification complexity. [source: https://arxiv.org/abs/2410.01794; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml]

Policy compliance and safeguard models

[fact] gpt-oss-safeguard and its Llama Guard comparator show that policy-conditioned safety classification is viable, but the repository-fit synthesis concluded that the open-weight safeguard family is better treated as a second-stage policy reasoner than as a first-line universal filter. [source: https://arxiv.org/abs/2312.06674; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]
[fact] The same synthesis also found that the documented model sizes and hardware expectations do not fit a default GitHub-hosted runner, which makes immediate inline deployment in research-review.yml the wrong first move. [source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]
[inference] For this repository, policy compliance should first become an explicit review criterion in research-review-prompt.md, with model-based policy classification reserved for a later backlog item once an external inference surface or sampled workflow exists. [source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]

Barnum statements and prompt design

[fact] The Barnum-statement synthesis defines the failure mode as low-specificity sentences that sound analytical while naming no concrete actor, mechanism, metric, disagreement, or decision consequence, and it recommends explicit review criteria rather than relying on generic anti-slop guidance alone. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md]
[fact] The same Barnum synthesis concludes that the safest automation path is layered: prompt-side specificity contracts first, cheap rule and specificity filtering second, and Large Language Model as judge escalation only for borderline cases. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]
[fact] The repository's earlier prompt-engineering synthesis found that prompt structure changes materially improved source engagement and finding quality, which supports upstream prevention as a high-leverage control surface for this item as well. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md]
[inference] research-prompt.md should therefore add a positive specificity contract, a short support-critical claim inventory, and an explicit recency note for live or fast-changing claims so that the review prompt is not asked to repair avoidable upstream defects. [source: https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://arxiv.org/abs/2310.05189]

Review-load and workflow-fit synthesis

[fact] The adjacent human-review-load synthesis found that universal pre-execution review degrades under volume and should be replaced by risk-tiered or exception-based patterns when human attention becomes the bottleneck. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.md]
[fact] The adjacent Large Language Model as judge synthesis found that the most stable operational pattern is layered evaluation, deterministic checks for exact structure or obvious failures first, semantic judging second, and governance or human escalation around the edges. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]
[inference] Those two adjacent items jointly support a deterministic pre-review lint step as the highest-impact new tooling addition, because it removes cheap-to-detect failures before the semantic review pass consumes scarce model and reviewer bandwidth. [source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]

Priority ordering and backlog drafting

[inference] The top prompt-only review change is to extend research-review-prompt.md Step 1 with an atomic-claim completeness check, Step 3 with an explicit Barnum-statement check, and Step 4 with a source-recency and time-sensitivity check for live claims. [source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://arxiv.org/abs/2310.05189]
[inference] The top low-infrastructure tooling change is to add a deterministic lint script before the Copilot review step in .github/workflows/research-review.yml, covering exact source-link parity, checked-source coverage, acronym expansion heuristics, time-sensitive-date checks, banned filler phrases, and Barnum-candidate concrete-anchor heuristics. [source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md]
[inference] The highest-value new backlog candidates are a deterministic review linter, a sampled support-critical claim verifier harness, and a Barnum-detector calibration pass over completed items. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md]

§3 Reasoning

[fact] Current-state claims are strongest when taken directly from research-review-prompt.md, research-prompt.md, and .github/workflows/research-review.yml, because those files define the repository's present review behavior. [source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml]
[fact] Tool-capability claims are strongest where the prerequisite items reached consistent conclusions, OpenFactCheck and FActScore for atomic-claim rigor, Loki for evidence-assistant workflows, gpt-oss-safeguard for policy-conditioned reasoning, and Barnum research for low-specificity output detection. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md]
[inference] The sequence prompt tightening -> deterministic lint -> sampled or optional semantic tooling is warranted because it matches both the repository's current workflow envelope and the layered-evaluation pattern already supported by adjacent completed work. [source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.md]
[inference] No reviewed source justifies immediate mandatory inline deployment of OpenFactCheck, FActScore, Loki, or gpt-oss-safeguard for every draft, because each tool family carries either corpus-bound assumptions, live-web dependence, infrastructure cost, or moderate transfer uncertainty. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

§4 Consistency Check

[fact] The evidence is consistent on one high-level pattern: the strongest reviewed systems all favor decomposition, layered evaluation, and human-readable intermediate evidence over one-shot whole-document verdicts. [source: https://arxiv.org/abs/2405.05583; https://arxiv.org/abs/2305.14251; https://arxiv.org/abs/2410.01794; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]
[fact] No direct contradiction surfaced between the Barnum synthesis and the FActScore or OpenFactCheck syntheses, because the former targets semantic emptiness while the latter target factual support and atomic precision. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]
[inference] The only live tension is between broader semantic checking and workflow simplicity, and the evidence resolves it in favor of staged adoption rather than all-at-once integration. [source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]

§5 Depth and Breadth Expansion

[inference] Technical lens: deterministic linting captures a large share of repeatable defects cheaply, while claim-level and policy-level model checks should be reserved for cases where deterministic parsing cannot decide the issue. [source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]
[inference] Economic lens: prompt rewrites and deterministic linting have lower operating cost than mandatory retrieval-heavy or model-heavy verification, so they dominate the short-term effort-to-impact ranking. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]
[inference] Behavioural lens: explicit Barnum and concrete-anchor checks matter because reviewers under load can accept fluent but low-information prose unless the workflow names that failure mode directly. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.md]
[inference] Governance lens: policy compliance belongs in the rubric now and in model-assisted enforcement later, because current standards and repository practice favor auditable layered controls over a single opaque semantic gate. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]

§6 Synthesis

Executive summary:

The next material improvement to this repository's automated review process should be a deterministic pre-review lint step plus explicit rubric additions for atomic-claim completeness and low-specificity generic-sounding sentences, not immediate mandatory inline deployment of OpenFactCheck, Loki, FActScore, or gpt-oss-safeguard. [inference; source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

research-review-prompt.md already checks citation binding, speculation control, prose quality, and logical coherence, but it does not make atomic-claim coverage, concrete-anchor absence, or source recency first-class review failures. [fact; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md]

The five prerequisite items converge on a layered architecture: prompt-side specificity contracts to prevent hollow prose, deterministic linting to catch repeatable defects cheaply, and sampled claim- or policy-level model assistance only where a bounded escalation path is justified. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]

A pragmatic sequence is therefore: rewrite research-review-prompt.md first, insert a deterministic lint step into research-review.yml second, strengthen research-prompt.md with a positive specificity contract third, and treat OpenFactCheck, FActScore, Loki, and safeguard-style policy classifiers as sampled or optional future backlog items rather than the immediate default gate. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

Key findings:

research-review-prompt.md currently enforces citation, speculation, prose-quality, and coherence checks, but it does not explicitly fail drafts for bundled multi-proposition claims, low-specificity generic-sounding sentences, or stale evidence on time-sensitive topics. ([fact]; medium confidence; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md)
The strongest prompt-only upgrade is to extend research-review-prompt.md Step 1 with an atomic-claim completeness check, Step 3 with an explicit check for Barnum statements, low-specificity sentences that sound analytical without adding decision-useful detail, and Step 4 with a recency check for live or fast-changing claims. ([inference]; high confidence; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://arxiv.org/abs/2310.05189)
The highest-impact low-infrastructure tooling change is a deterministic pre-review linter that checks exact source-link parity, checked-source coverage, acronym expansion, filler-phrase bans, concrete-anchor heuristics, and simple date-based recency rules before the semantic review pass runs. ([inference]; high confidence; source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md)
research-prompt.md should prevent defects upstream by requiring each analytical sentence to name a concrete actor, mechanism, metric, disagreement, or decision consequence and by adding a short support-critical claim inventory before final Findings are drafted. ([inference]; high confidence; source: https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md)
OpenFactCheck and FActScore both support a future support-critical claim-audit path, but the repository's own prior syntheses judge each tool family better as an offline diagnostic or sampled escalation path than as a mandatory per-item inline gate. ([inference]; medium confidence; source: https://arxiv.org/abs/2405.05583; https://arxiv.org/abs/2305.14251; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md)
Loki adds useful human-facing evidence discovery and gpt-oss-safeguard adds policy-conditioned reasoning, but both are better treated as later-stage or sampled integrations because Loki depends on live evidence retrieval and safeguard models do not fit the default GitHub-hosted review runner. ([inference]; medium confidence; source: https://arxiv.org/abs/2410.01794; https://arxiv.org/abs/2312.06674; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml)
The current evidence supports a pragmatic priority order of review-prompt rewrite first, deterministic lint second, research-prompt rewrite third, sampled support-critical claim verification fourth, and policy-classification or live-web evidence escalation after those foundations exist. ([inference]; medium confidence; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md])

Evidence map:

Claim	Source	Confidence	Notes
[fact] The current review prompt lacks explicit atomic-claim, Barnum, and recency failures.	https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md	medium	Direct prompt inspection
[inference] Adding explicit atomic-claim, Barnum, and recency checks is the strongest prompt-only upgrade.	https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://arxiv.org/abs/2310.05189	high	Low-cost rubric extension
[inference] A deterministic pre-review linter is the highest-impact low-infrastructure tooling change.	https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md	high	Cheap failures first
[inference] `research-prompt.md` should add a positive specificity contract and support-critical claim inventory.	https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md	high	Upstream prevention
[inference] OpenFactCheck and FActScore are better fits for offline or sampled claim verification than mandatory inline gating.	https://arxiv.org/abs/2405.05583; https://arxiv.org/abs/2305.14251; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md	medium	Prior-item conclusion convergence
[inference] Loki and safeguard models are useful later-stage aids, but not the first repository-default gate.	https://arxiv.org/abs/2410.01794; https://arxiv.org/abs/2312.06674; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml	medium	Workflow-fit constraint
[inference] A pragmatic effort-to-impact ordering is prompt rewrite, deterministic lint, prompt prevention, then sampled semantic tooling.	https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md	medium	Strategic synthesis

Assumptions:

Assumption: The repository will continue producing mixed-domain synthesis items rather than switching to one bounded verification corpus. Justification: the current prompts and completed corpus are oriented around multi-source synthesis rather than one canonical knowledge base. [assumption; source: https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]
Assumption: Barnum-candidate heuristics should begin as flags or warnings rather than hard failures until a repository-specific labeled sample exists. Justification: the Barnum synthesis found no direct benchmark for research-prose prevalence, so low-cost detection is justified before blocking enforcement is calibrated. [assumption; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://arxiv.org/abs/2310.13548]
Assumption: Time-sensitive claims can be identified with a simple author-supplied note or date heuristic before a more advanced retrieval layer exists. Justification: the current prompt already requires source enumeration, so adding a lightweight recency signal is operationally plausible without new infrastructure. [assumption; source: https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md]

Analysis:

The main trade-off is between coverage breadth and workflow friction, and the reviewed evidence favors staged controls rather than a single universal verifier. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.md]

For research-review-prompt.md, the exact low-cost change is to add one citation-discipline bullet for atomic-claim completeness, one remove-ai-slop bullet for Barnum statements and concrete-anchor absence, and one peer-reviewer bullet for stale or weakly dated evidence on time-sensitive claims. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]

Inside .github/workflows/research-review.yml, the exact tooling change is to run a deterministic lint script before the Copilot invocation so that missing checked sources, source-link mismatches, acronym misses, filler phrases, simple Barnum candidates, and date or recency omissions fail fast without consuming a full semantic review pass. [inference; source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]

Upstream in research-prompt.md, the exact change is to add a positive specificity contract near the Findings instructions and a short support-critical claim inventory near the end of investigation, because that combination prevents both low-specificity prose and over-bundled facts before review begins. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md]

The strongest rival remedy is to move directly to OpenFactCheck, FActScore, Loki, or safeguard-model integration, but that route is weaker as a first step because every reviewed tool family assumes a narrower corpus, heavier infrastructure, or a later-stage escalation workflow than the repository currently operates. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

That is why the priority matrix is asymmetric: review-prompt rewrite is low effort and high uplift, deterministic lint is low-to-medium effort and high uplift, research-prompt prevention is medium effort and high uplift, sampled claim verification is medium-to-high effort and medium uplift, and full policy or live-web semantic escalation is high effort with more conditional uplift. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

Risks, gaps, uncertainties:

Direct benchmark evidence for Barnum detection in research prose is still missing, so any first deterministic concrete-anchor rule should begin as a warning or low-severity failure until the repository calibrates it on labeled examples. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://arxiv.org/abs/2310.13548]
A simple source-recency rule can catch obviously stale citations, but it cannot by itself prove factual invalidity, so recency should qualify confidence and trigger review rather than serve as a stand-alone truth test. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://arxiv.org/abs/2310.05189]
Sampled support-critical claim auditing can improve factual precision, but it will still miss omission-heavy or non-sampled errors unless the repository later adds a bounded selection policy and calibration data. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]
Policy-model integration remains infrastructure-bound until the repository has an external inference surface or a clearly scoped sampled workflow, so policy compliance is best kept rubric-first in the short term. [inference; source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

Open questions:

Draft backlog item: Build deterministic research review linter - add a Python pre-review parser and checker that fails fast on source-link mismatches, unchecked seeded sources, acronym misses, filler phrases, simple Barnum candidates, and date or recency omissions before the Copilot review step. [inference; source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md]
Draft backlog item: Add support-critical claim audit harness - decompose Executive Summary and Key Findings into atomic claims and run sampled offline support checks inspired by FActScore and OpenFactCheck rather than a full-document inline gate. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]
Draft backlog item: Calibrate Barnum detection on the completed corpus - label a repository sample for concrete-anchor absence and compare deterministic heuristics with Large Language Model as judge scoring before making Barnum checks blocking. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]

§7 Recursive Review

[fact] Claim-label and inline-source audits were completed for this item before completion. [source: https://github.com/davidamitchell/Research/blob/main/Research/in-progress/2026-05-06-fact-checking-tools-research-quality-improvement.md]
[fact] Acronym expansion was checked for Large Language Model (LLM), Command Line Interface (CLI), and Uniform Resource Locator (URL). [source: https://github.com/davidamitchell/Research/blob/main/Research/in-progress/2026-05-06-fact-checking-tools-research-quality-improvement.md]
[fact] Cross-item scanning was repeated during drafting and §6-to-Findings parity was maintained. [source: https://github.com/davidamitchell/Research/blob/main/Research/in-progress/2026-05-06-fact-checking-tools-research-quality-improvement.md]
[inference] Overall confidence remains medium because the recommendations are multi-source and well supported, but the priority ordering remains a strategic synthesis rather than a benchmarked head-to-head comparison. [source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md]

Findings

Executive Summary

A pragmatic sequence is therefore: rewrite research-review-prompt.md first, insert a deterministic lint step into research-review.yml second, strengthen research-prompt.md with a positive specificity contract third, and treat OpenFactCheck, FActScore, Loki, and safeguard-style policy classifiers as sampled or optional future backlog items rather than the immediate default gate. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

research-review-prompt.md already checks citation binding, speculation control, prose quality, and logical coherence, but it does not make atomic-claim coverage, concrete-anchor absence, or source recency first-class review failures. [fact; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md]

The five prerequisite items converge on a layered architecture: prompt-side specificity contracts to prevent hollow prose, deterministic linting to catch repeatable defects cheaply, and sampled claim- or policy-level model assistance only where a bounded escalation path is justified. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]

The highest-value sequence is therefore: rewrite research-review-prompt.md first, insert a deterministic lint step into research-review.yml second, strengthen research-prompt.md with a positive specificity contract third, and treat OpenFactCheck, FActScore, Loki, and safeguard-style policy classifiers as sampled or optional future backlog items rather than the immediate default gate. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

Key Findings

research-review-prompt.md currently enforces citation, speculation, prose-quality, and coherence checks, but it does not explicitly fail drafts for bundled multi-proposition claims, low-specificity generic-sounding sentences, or stale evidence on time-sensitive topics. ([fact]; medium confidence; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md)
The strongest prompt-only upgrade is to extend research-review-prompt.md Step 1 with an atomic-claim completeness check, Step 3 with an explicit check for Barnum statements, low-specificity sentences that sound analytical without adding decision-useful detail, and Step 4 with a recency check for live or fast-changing claims. ([inference]; high confidence; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://arxiv.org/abs/2310.05189)
The highest-impact low-infrastructure tooling change is a deterministic pre-review linter that checks exact source-link parity, checked-source coverage, acronym expansion, filler-phrase bans, concrete-anchor heuristics, and simple date-based recency rules before the semantic review pass runs. ([inference]; high confidence; source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md)
research-prompt.md should prevent defects upstream by requiring each analytical sentence to name a concrete actor, mechanism, metric, disagreement, or decision consequence and by adding a short support-critical claim inventory before final Findings are drafted. ([inference]; high confidence; source: https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md)
OpenFactCheck and FActScore both support a future support-critical claim-audit path, but the repository's own prior syntheses judge each tool family better as an offline diagnostic or sampled escalation path than as a mandatory per-item inline gate. ([inference]; medium confidence; source: https://arxiv.org/abs/2405.05583; https://arxiv.org/abs/2305.14251; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md)
Loki adds useful human-facing evidence discovery and gpt-oss-safeguard adds policy-conditioned reasoning, but both are better treated as later-stage or sampled integrations because Loki depends on live evidence retrieval and safeguard models do not fit the default GitHub-hosted review runner. ([inference]; medium confidence; source: https://arxiv.org/abs/2410.01794; https://arxiv.org/abs/2312.06674; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml)
The current evidence supports a pragmatic priority order of review-prompt rewrite first, deterministic lint second, research-prompt rewrite third, sampled support-critical claim verification fourth, and policy-classification or live-web evidence escalation after those foundations exist. ([inference]; medium confidence; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md])

Evidence Map

Claim	Source	Confidence	Notes
[fact] The current review prompt lacks explicit atomic-claim, Barnum, and recency failures.	https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md	medium	Direct prompt inspection
[inference] Adding explicit atomic-claim, Barnum, and recency checks is the strongest prompt-only upgrade.	https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://arxiv.org/abs/2310.05189	high	Low-cost rubric extension
[inference] A deterministic pre-review linter is the highest-impact low-infrastructure tooling change.	https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md	high	Cheap failures first
[inference] `research-prompt.md` should add a positive specificity contract and support-critical claim inventory.	https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md	high	Upstream prevention
[inference] OpenFactCheck and FActScore are better fits for offline or sampled claim verification than mandatory inline gating.	https://arxiv.org/abs/2405.05583; https://arxiv.org/abs/2305.14251; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md	medium	Prior-item conclusion convergence
[inference] Loki and safeguard models are useful later-stage aids, but not the first repository-default gate.	https://arxiv.org/abs/2410.01794; https://arxiv.org/abs/2312.06674; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml	medium	Workflow-fit constraint
[inference] A pragmatic effort-to-impact ordering is prompt rewrite, deterministic lint, prompt prevention, then sampled semantic tooling.	https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md	medium	Strategic synthesis

Assumptions

Assumption: The repository will continue producing mixed-domain synthesis items rather than switching to one bounded verification corpus. Justification: the current prompts and completed corpus are oriented around multi-source synthesis rather than one canonical knowledge base. [assumption; source: https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]
Assumption: Barnum-candidate heuristics should begin as flags or warnings rather than hard failures until a repository-specific labeled sample exists. Justification: the Barnum synthesis found no direct benchmark for research-prose prevalence, so low-cost detection is justified before blocking enforcement is calibrated. [assumption; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://arxiv.org/abs/2310.13548]
Assumption: Time-sensitive claims can be identified with a simple author-supplied note or date heuristic before a more advanced retrieval layer exists. Justification: the current prompt already requires source enumeration, so adding a lightweight recency signal is operationally plausible without new infrastructure. [assumption; source: https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md]

Analysis

The main trade-off is between coverage breadth and workflow friction, and the reviewed evidence favors staged controls rather than a single universal verifier. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.md]

For research-review-prompt.md, the exact low-cost change is to add one citation-discipline bullet for atomic-claim completeness, one remove-ai-slop bullet for Barnum statements and concrete-anchor absence, and one peer-reviewer bullet for stale or weakly dated evidence on time-sensitive claims. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]

Inside .github/workflows/research-review.yml, the exact tooling change is to run a deterministic lint script before the Copilot invocation so that missing checked sources, source-link mismatches, acronym misses, filler phrases, simple Barnum candidates, and date or recency omissions fail fast without consuming a full semantic review pass. [inference; source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]

Upstream in research-prompt.md, the exact change is to add a positive specificity contract near the Findings instructions and a short support-critical claim inventory near the end of investigation, because that combination prevents both low-specificity prose and over-bundled facts before review begins. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md]

The strongest rival remedy is to move directly to OpenFactCheck, FActScore, Loki, or safeguard-model integration, but that route is weaker as a first step because every reviewed tool family assumes a narrower corpus, heavier infrastructure, or a later-stage escalation workflow than the repository currently operates. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

That is why the priority matrix is asymmetric: review-prompt rewrite is low effort and high uplift, deterministic lint is low-to-medium effort and high uplift, research-prompt prevention is medium effort and high uplift, sampled claim verification is medium-to-high effort and medium uplift, and full policy or live-web semantic escalation is high effort with more conditional uplift. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/research-prompt.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-loki-fact-checking-journalists-moderation.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

Risks, Gaps, and Uncertainties

Direct benchmark evidence for Barnum detection in research prose is still missing, so any first deterministic concrete-anchor rule should begin as a warning or low-severity failure until the repository calibrates it on labeled examples. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://arxiv.org/abs/2310.13548]
A simple source-recency rule can catch obviously stale citations, but it cannot by itself prove factual invalidity, so recency should qualify confidence and trigger review rather than serve as a stand-alone truth test. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://arxiv.org/abs/2310.05189]
Sampled support-critical claim auditing can improve factual precision, but it will still miss omission-heavy or non-sampled errors unless the repository later adds a bounded selection policy and calibration data. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]
Policy-model integration remains infrastructure-bound until the repository has an external inference surface or a clearly scoped sampled workflow, so policy compliance is best kept rubric-first in the short term. [inference; source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-gpt-oss-safeguard-policy-enforcement-open-weight.md]

Open Questions

Draft backlog item: Build deterministic research review linter - add a Python pre-review parser and checker that fails fast on source-link mismatches, unchecked seeded sources, acronym misses, filler phrases, simple Barnum candidates, and date or recency omissions before the Copilot review step. [inference; source: https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-03-03-research-loop-quality-prompt-engineering.md]
Draft backlog item: Add support-critical claim audit harness - decompose Executive Summary and Key Findings into atomic claims and run sampled offline support checks inspired by FActScore and OpenFactCheck rather than a full-document inline gate. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-02-automated-claim-verification-academic-literature.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md]
Draft backlog item: Calibrate Barnum detection on the completed corpus - label a repository sample for concrete-anchor absence and compare deterministic heuristics with Large Language Model as judge scoring before making Barnum checks blocking. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-28-llm-as-judge-pipeline-validation-checkpoints.md]

Output

Type: knowledge, backlog-item
Description: This item produces a prioritized upgrade path for the repository's automated review process, centered on review-rubric changes, a deterministic pre-review linter, and later sampled claim-verification or policy-classification backlog items rather than immediate heavyweight inline tool integration. [inference; source: https://github.com/davidamitchell/Research/blob/main/research-review-prompt.md; https://github.com/davidamitchell/Research/blob/main/.github/workflows/research-review.yml; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-openfactcheck-ai-fact-checking-pipeline.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-factscore-precision-scoring-atomic-claims.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-06-barnum-statements-ai-responses-theory-practice.md]
Links: