Skip to content

REQ 112

sergio velderrain edited this page Nov 11, 2026 · 2 revisions

REQ-112: Frame-addressable findings and sampled coverage

Requirement

The engine shall bind each image-based Ollama finding to the verified frames actually submitted for review and limit its reported coverage to that submitted sample.

Rationale

Category-level evidence references do not identify which image supports a claim, and three sampled images cannot establish absence of an event across a complete recording. Issue #233, under epic #229, adds attribution rather than changing actor capture production.

Fit Criterion

The host selects a bounded sample using a recorded deterministic policy/version and explicit maximum count. Request metadata binds run/capture identity, verified source-frame hash, unique frame index, and manifest timestamp when present; missing timestamps are recorded as unknown, never invented. For transformed images, metadata also binds the exact submitted-image hash and disclosure policy from REQ-113. Each image-based finding cites at least one submitted frame identity, and unknown, unreviewed, cross-run, or hash-mismatched references are rejected. Aggregation retains these references. The receipt records selected indices/hashes, reviewed count, available frame count, time coverage when known, and omitted coverage. It explicitly distinguishes sample-level observations from whole-capture conclusions, and does not claim an unreviewed interval or event was absent. The sampling policy is not presented as a guarantee of transient-event detection or temporal continuity. Model evidence and prompts remain untrusted data and cannot change the approved sample or execute tools.

Verification

Frame attribution supporting TEST-114 integrated through PR #241, implementation 69ab86be69ee5cf34239ef7cd83e4afebf4d7a17, merge 07bd285dae15d5b2fd0acafac7d01d58f666384d. The missing-reference reproducer failed before the fix; focused review/run tests passed 110 cases and the full Python suite passed 2,085 at integration. All seven upstream checks passed. Fixtures reject missing, duplicate, foreign-run, unreviewed and hash-substituted references; a synthetic off-sample event and absent timestamp remain explicitly unreviewed/unknown. References survive aggregation and privacy minimization. No whole-capture event-absence, continuous temporal coverage, real desktop inspection, or native-build proof is inferred.

Source

SYS-REQ-029; aggregation REQ-110 and privacy REQ-113

Priority

High

TestID

TEST-114

TestArtifact

scripts/tests/test_assisted_benchmark_review.py (test_image_findings_require_submitted_frame_references, test_image_findings_reject_unreviewed_or_duplicate_references, test_frame_identity_is_bound_to_run_and_submitted_image, test_sampling_reports_unreviewed_transient_and_unknown_time); scripts/tests/test_assisted_benchmark_run.py (sampling scope retained in the hashed receipt); .github/workflows/ci-assurance.yml. Run the focused review/run files or full scripts/tests/ through the isolated Python environment documented in tools/assisted-benchmark/README.md.

CodeRef

scripts/assisted_benchmark_review.py (select_evidence: run/index/time/source/submitted/policy frame identity; assessment_schema, build_review_request: prompt version 2 and request-scoped frame ID enum; validate_assessment: frame-reference enforcement; _aggregate_assessments, review_evidence: preserved attribution and explicit sample metadata); scripts/assisted_benchmark_run.py (orchestrate: reviewSampling in the hashed final receipt); tools/assisted-benchmark/README.md (sampling and coverage limitations).

Status

Implemented in develop; synthetic frame/citation/coverage regressions and CI verified, not continuous-capture or native-build qualification

Clone this wiki locally