Skip to content
This repository has been archived by the owner on Nov 1, 2022. It is now read-only.

Helm operater getting into upgrade, rollback loop when values change #2285

Closed
adrian opened this issue Jul 23, 2019 · 1 comment · Fixed by #2316
Closed

Helm operater getting into upgrade, rollback loop when values change #2285

adrian opened this issue Jul 23, 2019 · 1 comment · Fixed by #2316

Comments

@adrian
Copy link

adrian commented Jul 23, 2019

Describe the bug
If rollback is enabled and values are provided on a HelmRelease that causes an upgrade to fail (e.g. bad image tag), the operator will (correctly) roll the release back.

Shortly afterwards (I believe after helmOperator.chartsSyncInterval) the operator will do a dry-run upgrade to compare the values on the deployed release to those on the HelmRelease. These will be different since the deployed release will have the values that were rolled back to, while the HelmRelease will still have the new values. Consequently an upgrade will be kicked off which will fail and the loop continues again.

To Reproduce

  1. Create a HelmRelease setting rollback: true
  2. After the operator installs the release edit the HelmRelease and add values that will cause the release to fail, e.g. set a bad image tag (assuming the helm chart supports this parameter).

Expected behavior
The helm upgrade should fail. There should be no further activity on the release until the HelmRelease or helm chart are updated.

Logs
n/a

Additional context
Add any other context about the problem here, e.g

  • Helm Operator version: 0.10.0
  • Kubernetes version: v1.15.0
@adrian adrian added blocked-needs-validation Issue is waiting to be validated before we can proceed bug labels Jul 23, 2019
@stefanprodan stefanprodan added helm nice-to-have and removed blocked-needs-validation Issue is waiting to be validated before we can proceed bug labels Jul 25, 2019
@stefanprodan
Copy link
Member

Hi @adrian

This is the intended behaviour, the operator tries to reconcile the state in git with the cluster. A failure can happen from various reasons, for example you could make a cluster change (add a storage type, have more capacity, etc) that will make the release succeed, if helm-op would not retry the install then the release will get stuck forever.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants