Repository navigation
REQ 110
The engine shall aggregate validated per-input Ollama assessments without losing the evidence or blocking findings needed to justify the aggregate decision.
Appending findings and truncating the array can retain a failing verdict while discarding its supporting blocker. Issue #231, under epic #229, hardens aggregation after the initial evidence-scoped review feature in issue #226.
Any validated failing input makes the aggregate fail; an aggregate passes only when every required input was completed and passed, with no blocker or unresolved decision-relevant coverage gap. Other complete combinations are inconclusive. Malformed or missing input is an execution/validation failure, not an accepted model decision. Findings retain their supporting evidence identities under a documented deterministic ordering and deduplication rule. Output bounds prioritize blockers over informational detail; two claims with different evidence are not merged merely because their prose matches. If all required blocking support cannot fit, aggregation fails closed as incomplete rather than dropping support or returning pass. Any omitted non-blocking detail is counted and disclosed. The final aggregate is revalidated against its versioned schema and verdict/findings consistency rules; fail without a blocker and pass with a blocker are rejected. Deterministic capture/configuration/build failures are not inputs that a model verdict can reverse.
Regression coverage supporting TEST-112 integrated through PR #238, implementation commit 3e08defd28f8f093602f807c873829bb23745767, merge commit e1cde4b059fb75a48107ef75529b9544b2a10fee. The three-frame late-blocker reproducer failed against the earlier aggregator, then passed after the producer fix. Focused review/run tests pass 74 cases; the full offline Python suite passes 2,049 cases. All seven upstream PR checks passed, including the regression gate and source-config tests on Ubuntu and Windows. Fixtures cover fail precedence, incomplete/contradictory inputs, 32/33-blocker boundaries, omission counts, uncertainty downgrade, deterministic evidence-sensitive deduplication, and final revalidation. That aggregation change did not by itself establish frame attribution; REQ-112 now records its separate integrated evidence. No native build, continuous visual observation, or stable release is claimed.
SYS-REQ-029; complements frame attribution in REQ-112
High
scripts/tests/test_assisted_benchmark_review.py (test_aggregation_preserves_a_blocker_after_more_than_32_earlier_findings, test_aggregation_preserves_decision_precedence, test_aggregation_refuses_incomplete_or_contradictory_inputs, test_aggregation_refuses_blockers_that_exceed_the_output_bound, test_aggregation_retains_all_32_blockers_and_reports_optional_omissions, test_aggregation_uses_stable_severity_order_and_full_evidence_identity, test_aggregation_does_not_pass_when_uncertainties_are_present, test_aggregation_revalidates_its_final_schema_and_decision); scripts/tests/test_assisted_benchmark_run.py (aggregation policy/counts retained in the hashed run receipt). Reproduce with python3 -m pytest scripts/tests/test_assisted_benchmark_review.py scripts/tests/test_assisted_benchmark_run.py -q --tb=short and python3 -m pytest scripts/tests/ -q --tb=short. .github/workflows/ci-assurance.yml supplies the upstream regression gate.
Merged PR #238 implementation: scripts/assisted_benchmark_review.py (_aggregate_assessments: expected input count, severity ordering, canonical evidence-sensitive deduplication, blocker overflow refusal, final validation, and omission metadata; review_evidence: aggregate the validated required responses; validate_assessment and assessment_schema: retained output contract); scripts/assisted_benchmark_run.py (orchestrate: retain reviewAggregation in the hashed receipt); tools/assisted-benchmark/README.md (document projectbuddy/assessment-aggregation@1 and deterministic policy).
Implemented in develop; regression and CI verified through merged PR #238, not a stable-release or live-build qualification