feat(AF-499): Data Lifecycle Manager — retention policies, right-to-erasure, proxy-enforced pseudonymization & compliance#518
Merged
Conversation
…ation Add the lifecycle Spring Modulith module foundation: V103 schema (retention_policies, deletion_requests, decisions, lifecycle_runs, lifecycle_salt + enums), retention-policy CRUD + dry-run preview via the proxy QueryDryRunService, audit actions/resource types, and 7-locale i18n. Module boundaries, api-package purity, and i18n parity verified.
Add lifecycle_runs entity/repo, RetentionPolicyScanService (idempotent per-policy staging via dry-run eligibility), the clustered-safe RetentionPolicyScanJob (@scheduled + @SchedulerLock), and the LifecycleScanCompletedEvent. Docs (05/09) + CLAUDE.md env table updated.
Add deletion_requests + decisions persistence, the erasure state machine (PENDING_SCOPE_AI -> PENDING_REVIEW -> APPROVED/REJECTED/CANCELLED), self-service submit/list/cancel + admin review (approve/reject, no self-approval) endpoints, the async ErasureScopeAnalyzer (deterministic scope from retention policies; AI plug-in point), events, audit rows, and unit tests. API spec updated.
Contributor
Contributor
Add Testcontainers integration tests for the retention-policy controller (CRUD + preview + validation/404/403/401) and the erasure controllers (submit -> async scope -> approve/reject, self-approval 403, cancel, 404), exercising the controllers, DTOs, entities, repos, mappers, exception handler, page adapters, and audit writer end to end; plus a LifecyclePreviewCalculator unit test. Drop the surrounding @transactional on preview() so the proxy dry-run's datasource-lookup failure degrades to a best-effort estimate instead of poisoning the transaction.
Enabled PSEUDONYMIZE retention policies now contribute post-fetch ColumnMaskDirectives (LifecycleDirectiveResolutionService) merged into the query execution path by DefaultQueryLifecycleService, applied by the shared ColumnMasker. LifecycleTransform maps onto the masker; SHA256_SALTED/ TOKENIZATION use a per-org salt (LifecycleSaltService, AES-256-GCM in lifecycle_salt, rotatable) via a new backward-compatible salt param on the HASH strategy. Docs 05/07 updated.
… rewrite Add SoftDeleteDirective + IS_NULL RowSecurityOperator. RowSecurityRewriter now rewrites a plain DELETE on a soft-delete target into UPDATE SET marker=CURRENT_TIMESTAMP (fail-closed on DELETE...USING/joins), and an IS_NULL read filter hides soft-deleted rows from SELECT/UPDATE/DELETE. LifecycleDirectiveResolutionService resolves both from enabled SOFT_DELETE policies; DefaultQueryLifecycleService merges them onto the execution request (new softDeleteDirectives field, backward-compatible ctor). Docs 05 updated.
…liers
The new IS_NULL operator broke the exhaustive operator switches in the
mongodb/couchbase/elasticsearch/dynamodb row-security appliers. Translate it
to each engine's natural is-null/absent filter (Mongo {field:null}, N1QL IS
NULL, ES must_not exists, PartiQL IS MISSING) before the empty-values
deny-all guard, so a soft-delete read filter hides only deleted rows. Add
EsJson.exists + per-engine applier tests. Cassandra/Neo4j already fail-closed
via their default case.
Add the /admin/lifecycle/policies page (list + create modal + delete + dry-run preview), api/lifecycle.ts, lifecycle types + enum-label helpers, route + sidebar entry, and 7-locale i18n (parity green). Docs 06 + website feature card updated.
Contributor
Contributor
Coverage Report for Frontend Coverage (frontend)
File Coverage
|
||||||||||||||||||||||||||||||||||||||||||||
Add the self-service /lifecycle/erasure submit page (form + my-requests list with cancel) and the admin /admin/lifecycle/erasure review queue (optimistic approve/reject + scope-snapshot expander), erasureStatus tag-colour util + test, routes + sidebar entries, 7-locale i18n (parity green), and an e2e spec (seed via API, drive preview + approve via UI) with helpers. Docs 06 updated.
Add ComplianceReportType.RETENTION_ADHERENCE + RetentionAdherenceReportRow, a lifecycle.api LifecycleRunLookupService (compliance -> lifecycle.api, acyclic) exposing lifecycle_runs over a period, and the report builder + CSV/PDF writers + response mapping + GET /admin/compliance/reports/ retention-adherence (signed PDF/CSV export via the existing path). Backward- compatible ComplianceReport ctor; tests for the report path + lookup. Docs 04/05 updated.
Add tests for lifecycleActionLabel / lifecycleTransformLabel / lifecycleSubjectTypeLabel / erasureStatusLabel, raising enumLabels.ts function coverage (the CI per-file report flagged the new helpers as uncovered).
Wire an ERASURE_APPROVED NotificationEventType through the full fan-out: NotificationListener consumes ErasureRequestApprovedEvent (now carrying requestedBy) and dispatches via buildLifecycleErasure (submitter recipient) to the chat factories (Slack/Discord/Teams/Telegram) + in-app; no email template (fallback subject); PagerDuty not-applicable. notifications -> lifecycle.events (acyclic). Context-builder tests + docs 05/08.
Add ErasureExecutionService + clustered-safe ErasureExecutionJob: per scope table, run a governed parameter-bound DELETE FROM <table> through the proxy (subject predicate as a bound RowSecurityDirective; table validated as a simple identifier), reusing the SOFT_DELETE rewrite. Writes a lifecycle_runs row + tamper-evident DATA_ERASURE_COMPLETED proof-of-deletion audit, then transitions EXECUTED/FAILED (per-table failures isolated). New env var + scheduled-job docs. Unit tests cover happy path, failure, empty scope, and unsafe-identifier rejection.
Add the Data Lifecycle Manager to the README feature list, a 'Data lifecycle & right-to-erasure' feature card on the marketing homepage, a #cfg-lifecycle Configuration subsection (retention policies, erasure, env vars) on the operator docs page, and the content-source map row.
babltiga
marked this pull request as ready for review
June 29, 2026 15:56
babltiga
changed the title
… engines The IS_NULL row-security handling changed the engine appliers, so their reproducible shaded JARs drifted from the connector.json SHA-256 pins. Bump each plugin version (mongodb 1.0.5, couchbase 1.0.4, elasticsearch 1.0.3, dynamodb 1.0.2) and re-pin fileName/url/sha256. check-engine-pins.mjs green.
This was referenced Jul 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the Data Lifecycle Manager epic (#499) end-to-end: a new
lifecycleSpring Modulith module plus targeted extensions toproxy,workflow/ai,audit,compliance,notifications, and the engine plugins — with full frontend, e2e, docs, website, and 7-locale i18n.Built as verified, compiling, individually-green milestones (see commits). Opened as draft during construction; now complete.
What's implemented
/api/v1/lifecycle/policies, ADMIN, validated), targeting table/column-set/classification-tag with a window + action (HARD_DELETE/SOFT_DELETE/PSEUDONYMIZE); dry-run preview (impact without executing); clustered-safeRetentionPolicyScanJob.deletion_requestsstate machinePENDING_SCOPE_AI → PENDING_REVIEW → APPROVED → EXECUTED(+REJECTED/FAILED/CANCELLED); self-service submit + admin review (submitter can never self-approve); async scope detection (AI plug-in point).LifecycleTransform→ColumnMaskDirective, per-org AES-256-GCM salt with rotation); soft-delete read filter (newIS_NULLoperator) +DELETE → UPDATErewrite inRowSecurityRewriter(fail-closed). Engine appliers (mongo/couchbase/es/dynamo) handleIS_NULLnatively.ErasureExecutionJobruns governed, parameter-bound deletes per scope table through the proxy, reusing the soft-delete rewrite; writeslifecycle_runs+ tamper-evidentDATA_ERASURE_COMPLETEDproof-of-deletion audit;EXECUTED/FAILED.RETENTION_ADHERENCEreport (deletion history) overlifecycle_runs, signed PDF/CSV export.ERASURE_APPROVEDfan-out to the submitter across all chat channels + in-app./admin/lifecycle/policies(CRUD + preview),/lifecycle/erasure(submit),/admin/lifecycle/erasure(review queue, optimistic approve/reject); types, api, enum labels, routing, sidebar, 7-locale i18n.03–09, README, marketing homepage card + operator-docs section + content-source map; backend + frontend locale parity.Acceptance criteria
RetentionPolicyScanJobclustered-safe + idempotent + per-row tolerantDELETE→UPDATE+ read-time pseudonymization (fail-closed, no string-concat)SHA256_SALTED+ encrypted salt + rotation,FORMAT_PRESERVING,TOKENIZATION); aggregates preservedaudit_logrows + completion notifications03–09), README, website, i18n updated; locale + validation parity greenApplicationModulesTest+ApiPackageDependencyTestpass; coverage gates greenVerification
Backend: lifecycle suite +
ApplicationModulesTest+ApiPackageDependencyTest+MessagesParityTestgreen; engine appliers compile + pass. Frontend: lint + typecheck +test:coverage(974 tests, all thresholds met) + build green; locale parity green. E2E:lifecycle-manager.spec.ts(policy preview + erasure approve via UI).Known follow-ups (noted in docs)
subject_type).Closes #499.