Skip to content

feat(api: Improve deletion semantics for endpoints#669

Merged
iambriccardo merged 5 commits intomainfrom
new-reset-endpoint
Apr 20, 2026
Merged

feat(api: Improve deletion semantics for endpoints#669
iambriccardo merged 5 commits intomainfrom
new-reset-endpoint

Conversation

@iambriccardo
Copy link
Copy Markdown
Contributor

@iambriccardo iambriccardo commented Apr 17, 2026

Summary

Improve deletion behavior across the ETL API so destructive endpoints behave more predictably when pipelines are still active, and so source-side ETL state is cleaned up more deliberately during pipeline and tenant teardown.

What changed

  • Added active-pipeline guards to deletion endpoints so they return 409 Conflict instead of attempting deletes while a replicator is still active in Kubernetes.
  • Refactored pipeline deletion into smaller helpers that separate API-side cleanup, source-side ETL state removal, and replication slot deletion.
  • Updated source and destination delete endpoints to surface clearer conflict behavior when they are still referenced by pipelines.
  • Extended tenant deletion to clean source-side ETL state per source before deleting the tenant row in the API database.
  • Added Kubernetes helpers for determining whether a replicator should be treated as active for deletion guards.
  • Made replication slot deletion a no-op when there are no slots to remove.

Behavior changes

  • DELETE /pipelines/{id} now rejects deletion for active pipelines.
  • DELETE /destinations/{destination_id}/pipelines/{pipeline_id} now rejects deletion for active pipelines.
  • DELETE /sources/{id} and DELETE /destinations/{id} now return conflicts when the resource is still in use by pipelines.
  • DELETE /tenants/{tenant_id} now performs best-effort source cleanup first, then deletes the tenant row and lets Postgres cascade the tenant-owned API rows.
  • If a source database is already unreachable during tenant deletion, the request now logs the failure, skips source-side cleanup for that source, and still removes the tenant from the API database.

Tests

  • Updated deletion tests for pipelines, destinations, sources, and tenants to cover active vs. inactive pipeline behavior.
  • Added coverage for tenant/source uninstall paths, unreachable-source tenant deletion, and missing-ETL-table scenarios.
  • Added a Kubernetes unit test to ensure failed pods are treated as active for deletion guards.

Notes

This PR is primarily about deletion semantics and cleanup behavior. Most of the code churn is in route-level orchestration, shared deletion helpers, and the corresponding test coverage.

@iambriccardo iambriccardo changed the title new reset endpoint feat(api: Improve deletion semantics for endpoints Apr 17, 2026
@iambriccardo iambriccardo marked this pull request as ready for review April 17, 2026 12:43
@iambriccardo iambriccardo requested a review from a team as a code owner April 17, 2026 12:43
@coveralls
Copy link
Copy Markdown

coveralls commented Apr 17, 2026

Coverage Status

coverage: 78.197% (-0.05%) from 78.248% — new-reset-endpoint into main

Comment thread etl-api/src/routes/destinations_pipelines.rs Outdated
Comment thread etl-api/src/routes/tenants.rs Outdated
Comment thread etl-api/src/routes/tenants.rs Outdated
Comment thread etl-api/src/db/pipelines.rs Outdated
Comment thread etl-api/src/routes/tenants.rs
Comment thread etl-api/src/db/pipelines.rs Outdated
Comment thread etl-api/src/db/pipelines.rs
Comment thread etl-api/src/db/pipelines.rs Outdated
Comment thread etl-api/src/db/pipelines.rs Outdated
Comment thread etl-api/src/db/pipelines.rs Outdated
Comment thread etl-api/src/routes/tenants.rs
@iambriccardo iambriccardo requested review from bnjjj and farazdagi April 20, 2026 10:14
Copy link
Copy Markdown
Contributor

@farazdagi farazdagi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for looking into commented items. lgtm!

@iambriccardo iambriccardo merged commit 7b4eff6 into main Apr 20, 2026
13 checks passed
@iambriccardo iambriccardo deleted the new-reset-endpoint branch April 20, 2026 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants