Skip to content

Migration resources race condition: posit-team-system namespace may not exist #179

@stevenolen

Description

@stevenolen

Problem

The clusters step can fail when migration resources (ServiceAccount, Job) are created in the posit-team-system namespace before anything has created that namespace.

The Helm release for team-operator sets create_namespace=True, which would create posit-team-system — but the migration resources run before the Helm release, so the namespace doesn't exist yet when they're applied.

Relevant code: python-pulumi/src/ptd/pulumi_resources/team_operator.py

  • Migration ServiceAccount and Job target namespace=ptd.POSIT_TEAM_SYSTEM_NAMESPACE (~lines 99-201)
  • Helm release with create_namespace=True runs after (~line 321)
  • No explicit namespace creation or dependency exists for posit-team-system before the migration resources

Context

  • A fix was attempted in b074d4f (Feb 5) adding explicit namespace creation as a separate Pulumi resource
  • It was reverted in 7b64328 (Feb 6) because it conflicted with the Helm-managed namespace on running workloads
  • Other components (Loki, Mimir, Alloy, Grafana) all create their namespaces explicitly as separate Pulumi resources — team-operator is the exception

Possible fixes

  • Explicit namespace creation with adopt/import to avoid conflict with the existing Helm-managed namespace
  • Reorder dependencies so the Helm release (and its namespace creation) runs before migration resources
  • Use a Pulumi resource option (e.g., ignore_changes on the namespace) to prevent conflict on subsequent runs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions