* Add retention-for-training opt-in surface (v0.5.0)
Producer endpoints (rewrite, marks, impose, trap) and the CJD
orchestrator now honour an explicit opt-in signal — header
`X-Compile-Retain-For-Training: true` (or, on multipart endpoints,
form field `retain_for_training=true`) — that, combined with a
configured `COMPILE_RETAIN_BUCKET`, persists three blobs per call
(input.pdf, output.pdf, result.json) to S3-compatible storage with
a `ttl-days` object tag.
Default behaviour is unchanged: with no env config, the consent
header is parsed and logged but nothing is written. Backend failures
downgrade silently so a transient S3 outage never breaks a producer
call.
Surfaces:
* `compile_pdf.retention` — consent parser + tenant resolver + S3
store + `persist_if_opted_in` + `delete_by_sha256`.
* `POST /v1/retention/delete` — data-subject erasure endpoint.
* `LineageStep.retained_for_training` — flag threaded through CJD
and surfaced on GET /v1/lineage/{id}.
50 new tests across consent parsing, S3 store fakes, per-producer
wiring, CJD orchestration, retention API, and lineage round-trips.
Full suite: 465 passed.
https://claude.ai/code/session_01YQdUQcXxY93HLxjAfw2dRS
* Apply ruff format to retention test files
CI's `ruff format --check` step flagged tests/test_retention_cjd.py
and tests/test_retention_producer_apis.py. No behaviour change.
https://claude.ai/code/session_01YQdUQcXxY93HLxjAfw2dRS
---------
Co-authored-by: Claude <noreply@anthropic.com>