-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Problem
When EXTRACT_FIELD oracle contract validation fails, recovery sets value="extraction_failed" and confidence="low". This record propagates to Phase 5 synthesis. The SYNTHESIZE_CLAIM contract checks data_completeness and requires qualification for low-completeness papers — but there is no test verifying this end-to-end chain.
A plausible failure mode: a paper with multiple failed extractions gets cited in the review without qualification, making claims appear better-supported than the evidence warrants.
Proposed Fix
Add an integration test that:
- Creates a mock workspace with extraction records containing
extraction_failedvalues - Runs
validate_synthesis_claims()against a review that cites those papers without qualification - Verifies the validator catches the missing qualification
- Verifies the validator passes when qualification is present
Also: when Phase 4→3 feedback loop terminates at max_feedback_iterations with conceptual saturation still above θ_c, the progress line should include a warning — not just the neutral "returning to Phase 3" message. The current reporting understates the risk of an incomplete concept space.
Context
Leveson (STAMP/STPA Safety) panelist concern during panel review (2026-03-23). The individual pieces work (oracle contract recovery is tested, synthesis claim validation is tested) — the gap is the composition.