Intent observability: harden evidence storage, pagination, and rollups

[dinpd]

Parent: #45
Blocked by: #46, #60
Coordinates with: #62
Normative target: #58

Outcome

Remove prototype storage limits and make evidence queries, population accounting, and rollups complete, tenant-scoped, paginated, and reproducible at production scale.

Scope

• replace the small global audit index and bounded intent-evidence index
• tenant/time/run-aware append-only partitioning
• stable cursor pagination and immutable sorting
• explicit completeness, truncation, exclusion, and data-quality metadata
• idempotent record ingestion and conflicting-content rejection
• incremental or precomputed compatible-cohort rollups instead of full query-time scans
• separate query-selection filters from measurement-profile eligibility and exclusion rules
• record candidate_count, eligible_count, assessed_count, indeterminate_count, excluded_count, and missing_count
• publish explicit numerator/denominator pairs for every rate
• cohort keys that include measurement profile, intent profile when applicable, evaluator version, subject kind, finalization semantics, and interoperability version
• latest-final and as-of-cutoff selection for immutable assessment revisions
• retention and protected-artifact erasure behavior

Acceptance criteria

• No authoritative query silently truncates at the current approximate 250 audit or 2,000 evidence-entry limits.
• Evidence can be queried by tenant, run, intent, boundary action, decision, record type, and bounded time window.
• Pagination cannot duplicate or skip records within a stable result set.
• Replaying the same record_id is idempotent; conflicting content under one ID is rejected and audited.
• No global hot partition or cross-tenant scan is required for normal reads.
• Every rate exposes its numerator, denominator, candidate, eligible, assessed, indeterminate, excluded, and missing counts as applicable.
• Query filters do not silently redefine eligibility or exclusions.
• Rollups combine only compatible measurement, intent, evaluator, subject, finalization, and interoperability versions.
• Rollups are reproducible from immutable finalized assessments and can select the latest revision or an explicit as-of cutoff.
• Partial or unavailable sources are explicit in API and snapshot responses.
• Retention or artifact erasure does not falsify the remaining evidence record.

Validation

• scale tests exceeding existing index caps
• pagination stability and partition-isolation tests
• duplicate/conflict ingestion tests
• population-accounting invariants and numerator/denominator reconciliation
• incompatible-profile and evaluator-version isolation tests
• latest-versus-as-of revision rollup equivalence
• failure injection for partial indexes and unavailable artifacts

Non-goals

• changing assessment meaning
• combining unlike measurement, intent, evaluator, subject, or finalization versions