Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error on refresh for an unreachable cluster. #1522

Merged
merged 2 commits into from Apr 12, 2021

Conversation

lblackstone
Copy link
Member

Proposed changes

Previously, the refresh action would assume that an unreachable
cluster meant that the resources had been deleted out of band,
and would delete them from the state with a warning. This deletion
only affected the Pulumi state, not the actual k8s resource, and
could be repaired by reverting to the previous checkpoint.

Unfortunately, this behavior can lead to unintended state changes
in the case of a network partition, especially if the refresh operation
is running as part of an automated job.

This commit changes the default behavior of refresh to return an
error in the case of an unreachable cluster. This change means that
the state may have to be updated manually if the cluster was actually
deleted, but this seems like a preferable alternative to unexpected
state changes.

Related issues (optional)

Fix #1485

@lblackstone
Copy link
Member Author

This is also related to #416

If the cluster has actually been deleted, we may want to provide a way to force delete the associated resource state that doesn't involve manual editing.

Previously, the refresh action would assume that an unreachable
cluster meant that the resources had been deleted out of band,
and would delete them from the state with a warning. This deletion
only affected the Pulumi state, not the actual k8s resource, and
could be repaired by reverting to the previous checkpoint.

Unfortunately, this behavior can lead to unintended state changes
in the case of a network partition, especially if the refresh operation
is running as part of an automated job.

This commit changes the default behavior of refresh to return an
error in the case of an unreachable cluster. This change means that
the state may have to be updated manually if the cluster was actually
deleted, but this seems like a preferable alternative to unexpected
state changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Invalid stack state stored when a K8s cluster is not responding during refresh?
2 participants