Skip to content

fix(team-operator): add retain_on_delete protection for CRDs and namespace#90

Merged
timtalbot merged 3 commits intomainfrom
fix-crd-retain-on-delete
Jan 28, 2026
Merged

fix(team-operator): add retain_on_delete protection for CRDs and namespace#90
timtalbot merged 3 commits intomainfrom
fix-crd-retain-on-delete

Conversation

@ian-flores
Copy link
Contributor

Summary

When migrating from kustomize to Helm deployment, Pulumi sees old kustomize-managed CRD resources as orphaned and deletes them. This causes cascade deletion of Site custom resources - as happened in npower01-production.

This fix adds protection to prevent Pulumi from deleting critical resources during migration:

  • Protected CRDs: Creates CRD resources with retain_on_delete=True and ignore_changes=["*"]
  • Aliases: Links to old kustomize resource URNs so Pulumi recognizes existing CRDs
  • Skip CRDs in Helm: Sets skip_crds=True on Helm release to avoid conflicts
  • Protected namespace: Adds retain_on_delete=True to posit-team namespace

Root Cause

The migration from kustomize to Helm changed how CRDs are managed:

  • Old code: kustomize.Directory created CRDs as individual Pulumi resources
  • New code: Helm chart manages CRDs internally (not as separate Pulumi resources)
  • Result: Pulumi sees old CRD resources as orphaned → deletes them → cascade deletes Sites

The helm.sh/resource-policy: keep annotation only protects against Helm-initiated deletion, not Pulumi-initiated deletion.

Test plan

  • Run ptd ensure on a workload that hasn't migrated yet - verify CRDs are protected
  • Run pulumi preview to verify no unexpected deletions
  • Verify Sites survive migration

🤖 Generated with Claude Code

…space

When migrating from kustomize to Helm deployment, Pulumi would see old
kustomize-managed CRD resources as orphaned and delete them, causing
cascade deletion of Site custom resources.

This fix:
- Creates protected CRD resources with retain_on_delete=True
- Uses aliases to link to old kustomize resource URNs
- Sets skip_crds=True on Helm release to avoid conflicts
- Adds ignore_changes=["*"] so Pulumi doesn't modify CRDs
- Adds retain_on_delete=True to posit-team namespace

Fixes cascade deletion issue seen in npower01-production migration.
@ian-flores ian-flores force-pushed the fix-crd-retain-on-delete branch from 1bff1bc to 8c2d8ce Compare January 22, 2026 19:02
Remove CRD protection code that was causing conflicts with existing CRDs.
Keep only retain_on_delete=True on namespace.

CRD protection needs more work to handle existing CRDs properly.
Add configuration option to skip CRD installation during Helm deployment,
enabling safe migration from kustomize-managed CRDs. When enabled:
- Sets crd.enable=false in Helm values (prevents template rendering)
- Sets skip_crds=true on ReleaseArgs (skips crds/ directory)
- Keeps crd.keep=true as defense-in-depth

This allows the migration job to patch existing CRDs with Helm ownership
labels without risk of accidental deletion during the kustomize-to-Helm
transition.
@ian-flores ian-flores closed this Jan 27, 2026
@ian-flores ian-flores reopened this Jan 28, 2026
@ian-flores ian-flores marked this pull request as ready for review January 28, 2026 17:34
@ian-flores ian-flores requested a review from a team as a code owner January 28, 2026 17:34
@ian-flores ian-flores requested a review from timtalbot January 28, 2026 17:34
@timtalbot timtalbot merged commit 4a0a98e into main Jan 28, 2026
3 checks passed
@timtalbot timtalbot deleted the fix-crd-retain-on-delete branch January 28, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants