Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: unable to delete an Application if its target cluster is deleted, Argo CD enters infinite app deletion reconciliation loop #6557

Merged
merged 1 commit into from
Jun 29, 2021

Conversation

jgwest
Copy link
Member

@jgwest jgwest commented Jun 24, 2021

Description

On Application deletion, this PR verifies whether the target cluster (defined in the destination field of the Application resource) is defined within Argo CD (exists as a cluster secret, or is a local cluster). If the target cluster is not defined, it allows the Application to be deleted (rather than returning an error, as before).

Details

If the target cluster is not defined, it is assumed that either:
a) the cluster was deleted from Argo CD (for example, via the Argo CD settings UI, or by deleting the cluster secret)
or
b) the Application spec is invalid (referencing a cluster that doesn't exist).

In both cases, since there are no cluster resources to clean up, we can safely allow the Application resource to be deleted.

So, if we detect that the Application targets a valid cluster:

  • No change: everything works in this function as it currently does.

OTOH, if we detect the Application contains an invalid cluster:

  • We empty the various Argo CD caches for the Application, as before.
  • We skip the steps that would delete the live resources from the target cluster (because there is no target cluster)
  • We report a warning, but allow the Application resource to be deleted (eg we remove the finalizer from the Application)

See parent issue for reproduction steps and details.

Fixes #5817

Note on DCO:

If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • Optional. My organization is added to USERS.md.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).

…, Argo CD enters infinite app deletion reconciliation loop

Signed-off-by: Jonathan West <jonwest@redhat.com>
@codecov
Copy link

codecov bot commented Jun 24, 2021

Codecov Report

Merging #6557 (3906c5b) into master (9815490) will increase coverage by 0.12%.
The diff coverage is 42.22%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6557      +/-   ##
==========================================
+ Coverage   41.14%   41.27%   +0.12%     
==========================================
  Files         153      156       +3     
  Lines       20313    20677     +364     
==========================================
+ Hits         8358     8534     +176     
- Misses      10796    10936     +140     
- Partials     1159     1207      +48     
Impacted Files Coverage Δ
pkg/apis/application/v1alpha1/types.go 57.89% <ø> (ø)
util/argo/argo.go 61.75% <ø> (ø)
controller/appcontroller.go 53.36% <42.22%> (+0.09%) ⬆️
util/db/repository.go 40.00% <0.00%> (-23.30%) ⬇️
util/db/cluster.go 60.00% <0.00%> (-3.34%) ⬇️
util/db/repository_legacy.go 48.87% <0.00%> (ø)
util/db/repository_secrets.go 67.52% <0.00%> (ø)
util/db/secrets.go 78.08% <0.00%> (ø)
server/application/logs.go 85.71% <0.00%> (+1.23%) ⬆️
server/rbacpolicy/rbacpolicy.go 83.33% <0.00%> (+4.30%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9815490...3906c5b. Read the comment docs.


filteredObjs := FilterObjectsForDeletion(objs)
objsMap, err := ctrl.getPermittedAppLiveObjects(app, proj)
if err != nil {
Copy link
Member Author

@jgwest jgwest Jun 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reviewers: FYI the code inside this if block is unchanged from the original (there aren't any changes hidden in there 😄 ), with the only exception being moving the call to ctrl.db.GetCluster to above the block.

@jgwest jgwest marked this pull request as ready for review June 24, 2021 17:20
Copy link
Member

@jannfis jannfis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @jgwest !

Before you merge this PR, please assign it to yourself.

Comment on lines +1570 to +1575
// This call to 'ValidateDestination' ensures that the .spec.destination field of all Applications
// returned by the informer/lister will have server field set (if not already set) based on the name.
// (or, if not found, an error app condition)

// If the server field is not set, set it based on the cluster name; if the cluster name can't be found,
// log an error as an App Condition.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines +2406 to +2411
// An ApplicationDestination has an 'inferred server' if the ApplicationDestination
// contains a Name, but not a Server URL. In this case it is necessary to retrieve
// the Server URL by looking up the cluster name.
//
// As of this writing, looking up the cluster name, and setting the URL, is
// performed by 'utils.ValidateDestination(...)', which then calls SetInferredServer.
Copy link
Member

@jannfis jannfis Jun 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines +292 to +300
// ValidateDestination sets the 'Server' value of the ApplicationDestination, if it is not set.
// NOTE: this function WILL write to the object pointed to by the 'dest' parameter.
//
// If an ApplicationDestination has a Name field, but has an empty Server (URL) field,
// ValidationDestination will look up the cluster by name (to get the server URL), and
// set the corresponding Server field value.
//
// It also checks:
// - If we used both name and server then we return an invalid spec error
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@jgwest
Copy link
Member Author

jgwest commented Jul 29, 2021

Previously verified downstream: argoproj/applicationset#175 (comment) so removing tag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants