Milestone 5 design: exports and recovery

[TusanHomichi]

Problem or outcome

Milestone 4 closed with #32: the record substrate is complete — canonical bytes, immutable versions, acknowledgments, amendments, weekly summaries, task signoffs. Milestone 5 (docs/roadmap.md) makes the installation leave-able and recoverable: deterministic PDF records, complete structured exports, trainee packets, retention-policy administration with holds and lawful disposition, scheduled backup retention, restore verification, operator documentation, and the operator interface for those workflows. Its exit: a center can leave with all of its data and can prove recovery from a clean installation.

This issue records the design decisions before implementation, in the #22/#32 pattern. Design targets already fixed by the corpus: exports round-trip (docs/records-integrity.md), Typst is the planned PDF renderer and a PDF is a presentation of a version, never the record (docs/architecture.md), disposition follows the five-step flow with typed holds and tombstone DispositionEvents, and backups are VACUUM INTO snapshots with validation, durability, retention, and a tested restore workflow.

The decisions

1. Structured record-export identity. What a record export is. Recommendation: the stored canonical bytes travel verbatim — never re-serialized — beside a manifest (instance id, record and version identity, record_schema, both hashes, predecessor hash, export instant, format version), one documented format under docs/formats/ like the program export; multi-record exports are an archive of the same units plus an archive manifest. Schema-1 and schema-2 bytes coexist as they are stored; verification of an export needs nothing but the export.

2. PDF determinism and scope. Whether PDFs land in this milestone and what "deterministic" commits to. Recommendation: in scope as a late slice; Typst pinned with shipped redistribution-friendly fonts; templates render only from the stored envelope; determinism proven by fixture comparison within documented tolerances (the roadmap's own phrasing), never claimed as byte-exactness across Typst upgrades — the structured export stays the record of authority.

3. Trainee packet contents. Recommendation: everything retained about one enrollment: every retained version of every record (superseded originals included), acknowledgments, amendment records, the signoff history, structured units plus their PDFs, one packet manifest. Nothing summarized-only; the packet is the trainee leaving with their records.

4. Retention, holds, and disposition model. Recommendation: retention policy as versioned configuration (record class, disposition authority, trigger, period, action — the records-integrity fields), the typed hold set from docs/records-integrity.md, and the five-step disposition flow exactly as written: policy check → hold check → an authorized operator reviews the exact scope and authority → destruction of content, attachments, snapshots, and in-scope derived exports → tombstone DispositionEvent with independent retention. A new explicit capability gates it (PRINCIPLES.md 10: broader administration stays explicit authority); no bundle carries it by default. Chain verification learns to report policy-boundary closure honestly.

5. Backup retention and restore proof. Recommendation: keep the existing snapshot pipeline; retention as configured count-plus-age pruning; restore verification as a product workflow (restore --verify and an operator page): restore into a scratch directory, run migrations forward, recompute every stored version's content and chain hashes, and report — recovery is proven, not assumed, matching AGENTS.md ("a passing unit test is not recovery proof").

6. Operator-interface scope for v1. Recommendation: web interface for exports, packets, and read-only retention/backup status; disposition preview-and-confirm in the web behind the new capability with the typed scope shown before the act; backup/restore drills remain CLI-first this milestone (they operate on the data directory beneath the server), with the operator documentation covering both.

Slice plan (once decisions settle)

1. Structured exports and the export format document (single records, then archives).
2. Trainee packets over the export units.
3. Retention policy configuration, holds, disposition with tombstones, and the new capability.
4. Backup retention, restore verification, PDFs (Typst), and operator documentation/interface polish.

Related roadmap, decisions, or issues

• docs/roadmap.md Milestone 5; docs/records-integrity.md (exports, retention/holds/disposition, verification obligations); docs/architecture.md (Typst, backups, data directory); PRINCIPLES.md
• Builds on Milestone 4 design: defensible records (owner decisions needed) #32's substrate; ADRs 0011–0013 fix what exports carry
• Standing related items: Write transactions: deferred snapshots can surface SQLITE_BUSY_SNAPSHOT as 500s under concurrent writes #27 (deferred-transaction retrofit), Draft workspace: keep the losing writer's text visible after a stale-save reload #34 (stale-save UX)

[TusanHomichi]

Decisions recorded. The project owner approved all six recommendations as written ("Let's go with your recommendations and continue"):

1. Structured exports carry the stored canonical bytes verbatim beside a manifest (instance identity, record and version identity, record_schema, content hash, chain hash, predecessor hash, export instant, format version); archives are units plus an archive manifest; the format is documented under docs/formats/.
2. PDFs come from a pinned Typst toolchain in a late slice, deterministic within documented tolerances; the structured export stays the authority.
3. Trainee packets contain everything retained: every version including superseded ones, acknowledgments, amendments, signoff history, PDFs, and a packet manifest.
4. Retention policy is versioned configuration; holds are a typed set; disposition is the five-step flow with a tombstone DispositionEvent under independent retention, a new explicit capability, and chain verification that reports policy-boundary closure.
5. Backup retention prunes by count and age; restore --verify restores to a scratch directory, migrates, and recomputes every version's hashes.
6. Exports, packets, and read-only status get web UI; disposition is preview-and-confirm behind its capability; backup and restore stay CLI-first.

Build proceeds in the slice order above, starting with slice 1 (structured exports and the export format document). Each slice gets its own issue referencing this one.

---

Generated by Claude Code