Feature Request: helm delete --wait #2378

ahawkins · 2017-05-03T06:52:57Z

helm install --wait can wait for everything to be ready. Can a new helm delete --wait do something similar? I'm using helm delete --purge in my build pipeline followed by this script (to do something similar to --wait).

#!/usr/bin/env bash

set -euo pipefail

main() {
	local kinds=( Service Deployment Pod Secret ConfigMap )
	local counter=0 attempts=30

	for release in "$@"; do
		echo "--> Deleting release ${release}"

		if helm ls --kube-context "${KUBE_CONTEXT}" | grep -qF "${release}"; then
			echo "==> Found helm release; deleting with --purge"
			helm delete "${release}" --purge  --kube-context "${KUBE_CONTEXT}"
		else
			echo "==> No release found; deleting manually"

			for kind in "${kinds[@]}"; do
				echo "==> Deleting any dangling ${kind}"

				kubectl delete "${kind}" \
					-l "release=${release}" \
					-n "${KUBE_NAMESPACE}" \
					--force \
					--grace-period 0 \
					--context "${KUBE_CONTEXT}" 2>/dev/null
			done
		fi

		echo "--> Awaiting resource deleting confirmation"
		for kind in "${kinds[@]}"; do
			counter=0

			while [ $counter -lt $attempts ]; do
				pending_resources="$(kubectl get "${kind}" \
					-o wide \
					-l "release=${release}" \
					-n "${KUBE_NAMESPACE}" \
					--context "${KUBE_CONTEXT}" 2>/dev/null
				)"

				if [ -n "${pending_resources}" ]; then
					echo "${release} ${kind} still running. ${counter}/${attempts} tests completed; retrying."
					echo "${pending_resources}" 1>&2
					echo 1>&2

					# NOTE: The pre-increment usage. This makes the arithmatic expression
					# always exit 0. The post-increment form exits non-zero when counter
					# is zero. More information here: http://wiki.bash-hackers.org/syntax/arith_expr#arithmetic_expressions_and_return_codes
					((++counter))
					sleep 10
				else
					break
				fi
			done

			if [ $counter -eq $attempts ]; then
				echo "${release} ${kind} failed to delete in time.";
				return 1
			fi
		done

		echo "--> Awaiting helm confirmation"
		counter=0

		while [ $counter -lt $attempts ]; do
			if helm ls --all --kube-context "${KUBE_CONTEXT}" | grep -qF "${release}"; then
				echo "${release} still in tiller. ${counter}/${attempts} checks completed; retrying."

				# NOTE: The pre-increment usage. This makes the arithmatic expression
				# always exit 0. The post-increment form exits non-zero when counter
				# is zero. More information here: http://wiki.bash-hackers.org/syntax/arith_expr#arithmetic_expressions_and_return_codes
				((++counter))
				sleep 10
			else
				break
			fi
		done

		if [ $counter -eq $attempts ]; then
			echo "${release} failed to purge from tiller delete in time.";
			return 1
		fi
	done
}

main "$@"

The text was updated successfully, but these errors were encountered:

michelleN · 2017-05-03T15:04:04Z

There needs to be some refactoring of the --wait logic before this can happen but yeah it makes sense.

alisondy · 2018-08-12T03:09:46Z

/remove-lifecycle rotten

alisondy · 2018-08-12T03:10:15Z

/remove-lifecycle stale

Without helm/helm#2378 there isn't a way to deterministically know when all resources are finished being removed. A simple thing we can do is warn the user on a ES_FORCE_INSTALL=true.

umomany · 2019-05-08T11:21:55Z

/remove-lifecycle rotten

umomany · 2019-05-08T11:22:01Z

/remove-lifecycle stale

bacongobbler · 2019-05-08T16:51:04Z

We removed the stale bot from helm/helm after the move to the CNCF. No need to try and remove the stale labels; it's all been removed. :)

YingjunHu · 2019-07-19T18:37:21Z

Any luck on this?

bacongobbler · 2019-08-14T21:39:54Z

Nope. Are you working on an implementation?

YingjunHu · 2019-08-14T21:56:56Z

Not yet since I wasn't sure if there's any ongoing fix. I'll try to implement it if it's not planned for both helm2 and 3

bacongobbler · 2019-08-14T21:57:49Z

AFAIK nobody's working on this. Feel free to work on an implementation!

bacongobbler · 2020-08-19T16:28:23Z

Is the intention here that order would be respected too?

I would assume so, yes. We'd need to delete objects in reverse order. Otherwise we may delete its parent and report a false positive that the object no longer exists (e.g. by deleting a ClusterRole before deleting a ClusterRoleBinding).

bacongobbler · 2020-08-19T16:30:08Z

Re-marking this as a feature; this has been a highly requested feature for some time. If someone from the community wishes to implement this, feel free.

github-actions · 2020-11-22T00:30:41Z

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

Bessonov · 2020-11-22T01:02:55Z

Activity :/

bacongobbler · 2020-12-16T19:26:57Z

In the interest of preventing spam to users subscribing to this thread, instead of commenting "+1" or "waiting for it", please add a 👍 to the OP or consider hitting the "subscribe" button on the top-right, under "Notifications". That way we can keep the conversation relevant to the discussion at hand. I've gone ahead and deleted those comments. Thanks!

bacongobbler · 2020-12-16T19:36:12Z

That being said, I'm interested to hear who from the community is willing to work on this feature. OP proposed this idea over three and a half years ago.

There has to be a point where we say the community has not taken enough initiative on this feature request, and we have to close it out as stale/unimplemented.

@omerfsen @arocki7 @micktg any takers?

Bessonov · 2020-12-16T20:17:04Z

@bacongobbler I don't think that closing this issue is the right way to handle it. If you close it, you get new issues about this topic. Just remove the stale bot, then everyone get less spam. A bot against "+1" spammers would do better job.

jerriais · 2021-01-19T09:30:43Z

Hi @bacongobbler
I am working on this now.
When I try to delete a deployment, I want to wait the pods controlled by the deployment until they all stop.
However those pods don't appear in the big chart manifest, I have no idea which label or selector I can use.
Any suggestions?

When I'm doing this manually I often find that even when all pods are gone some services are still being deleted watch kubectl get svc. Maybe it could work by checking to see if all services are gone, then the delete can be considered as complete...

bacongobbler · 2021-01-19T16:40:43Z

I don't think that closing this issue is the right way to handle it. If you close it, you get new issues about this topic.

The Helm project receives about 3-4 new tickets per day. Nearly all of them are support issues or requests for clarification. The stale issue bot has been incredibly helpful to manage these issues, as many of them are either left open or fall off our radar, either because there's little engagement from the OP or the community on certain tickets. On the off chance that there is a ticket that needs to stay open, we can add "keep open" labels to the ticket.

We are very keen to review community contributions that are willing to take the time to implement this feature. But if the ticket goes stale due to 3 months of inactivity (and in this case, it's been left open for nearly 4 years), then it's clearly not a priority ticket item, or community members have found a workaround (as the OP demonstrated in his first comment) and there's little incentive to implement the requested feature.

archerbj · 2021-03-09T07:10:59Z

looking forward to seeing this

Ornias1993 · 2021-03-13T10:00:07Z

I don't think that closing this issue is the right way to handle it. If you close it, you get new issues about this topic.

The Helm project receives about 3-4 new tickets per day. Nearly all of them are support issues or requests for clarification. The stale issue bot has been incredibly helpful to manage these issues, as many of them are either left open or fall off our radar, either because there's little engagement from the OP or the community on certain tickets. On the off chance that there is a ticket that needs to stay open, we can add "keep open" labels to the ticket.

We are very keen to review community contributions that are willing to take the time to implement this feature. But if the ticket goes stale due to 3 months of inactivity (and in this case, it's been left open for nearly 4 years), then it's clearly not a priority ticket item, or community members have found a workaround (as the OP demonstrated in his first comment) and there's little incentive to implement the requested feature.

Just because no one submits a fix, doesn't mean no one would willing to. It only means what it says on the tin: No one submitted a fix.
Assuming willingness was the issue is a retorical mistake. It could just as well be simply the users having need for this issue, not being comfortable or knowlegable enough to fix it, them not having the time required to put into it, etc.

For example:
I personally would love this feature but I:
A. Am not comfortable or knowlegable with your codebase at all
B. I don't find myself having the required development skillset for this kind of PR
C. Don't have the time to put into it besides 40+ hours a week on opensource already

I would love to put it in, if I had the hunderd+ hours available to learn your codebase and find out a way to put it in.

ishaan-rd · 2021-05-08T15:27:48Z

Any updates?

eranelbaz · 2021-07-11T13:02:58Z

Got this issue...
uninstalled helm chart but all the resources was still up...
I had to remove them using kubectl cli and search for all resources by hand...

patsevanton · 2021-07-13T08:20:36Z

I use werf - https://werf.io/ - https://github.com/werf/werf
werf helm uninstall application

If set, 'uninstall' command will wait until all the resources are deleted before returning. It will wait for as long as --timeout closes helm#2378 Signed-off-by: Mike Ng <ming@redhat.com>

michelleN added the proposal label May 3, 2017

k8s-ci-robot added the lifecycle/stale label Dec 24, 2017

k8s-ci-robot added lifecycle/rotten and removed lifecycle/stale labels Jan 23, 2018

k8s-ci-robot closed this as completed Feb 22, 2018

michelleN removed the lifecycle/rotten label Feb 26, 2018

michelleN reopened this Feb 26, 2018

k8s-ci-robot added the lifecycle/stale label May 27, 2018

k8s-ci-robot added lifecycle/rotten and removed lifecycle/stale labels Jun 26, 2018

bacongobbler removed the lifecycle/rotten label Aug 12, 2018

jsravn mentioned this issue Oct 15, 2018

Warning on force_install that it may fail lightbend/console-charts#147

Merged

Jancis mentioned this issue Nov 16, 2018

SLT-169: Removes failed release before deployment if it was the first release wunderio/drupal-project-k8s#67

Merged

helm deleted a comment from fejta-bot May 8, 2019

bacongobbler added feature and removed proposal labels Aug 19, 2020

LukeCarrier mentioned this issue Sep 30, 2020

Add option to delete the namespace created during install #1464

Closed

bacongobbler mentioned this issue Nov 9, 2020

helm delete is not honoring --timeout, it exists even before that #7221

Closed

github-actions bot added the Stale label Nov 22, 2020

github-actions bot removed the Stale label Nov 26, 2020

helm deleted a comment from mickdewald Dec 16, 2020

helm deleted a comment from arocki7 Dec 16, 2020

helm deleted a comment from omerfsen Dec 16, 2020

aareet mentioned this issue Jan 6, 2021

Terraform destroy of helm_release resources. hashicorp/terraform-provider-helm#593

Closed

bacongobbler mentioned this issue Apr 9, 2021

helm uninstall garbage collection PropagationPolicy #9572

Closed

mikeshng mentioned this issue May 15, 2021

feat: add optional boolean '--wait' flag to 'uninstall' command. #9702

Merged

3 tasks

mattfarina closed this as completed in #9702 Aug 30, 2021

juliayakovlev mentioned this issue Oct 17, 2021

test(functional_test): deploy helm chart with all default values scylladb/scylla-cluster-tests#4012

Merged

7 tasks

bianchi2 mentioned this issue Nov 10, 2022

CLIP-1701: Wait until pods in Helm release are terminated before destroying nfs module atlassian-labs/data-center-terraform#282

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature Request: helm delete --wait #2378

Feature Request: helm delete --wait #2378

ahawkins commented May 3, 2017

michelleN commented May 3, 2017

alisondy commented Aug 12, 2018

alisondy commented Aug 12, 2018

umomany commented May 8, 2019

umomany commented May 8, 2019

bacongobbler commented May 8, 2019

YingjunHu commented Jul 19, 2019

bacongobbler commented Aug 14, 2019

YingjunHu commented Aug 14, 2019

bacongobbler commented Aug 14, 2019

bacongobbler commented Aug 19, 2020 •

edited

bacongobbler commented Aug 19, 2020

github-actions bot commented Nov 22, 2020

Bessonov commented Nov 22, 2020

bacongobbler commented Dec 16, 2020

bacongobbler commented Dec 16, 2020 •

edited

Bessonov commented Dec 16, 2020

jerriais commented Jan 19, 2021

bacongobbler commented Jan 19, 2021 •

edited

archerbj commented Mar 9, 2021

Ornias1993 commented Mar 13, 2021 •

edited

ishaan-rd commented May 8, 2021

eranelbaz commented Jul 11, 2021

patsevanton commented Jul 13, 2021 •

edited

Feature Request: helm delete --wait #2378

Feature Request: helm delete --wait #2378

Comments

ahawkins commented May 3, 2017

michelleN commented May 3, 2017

alisondy commented Aug 12, 2018

alisondy commented Aug 12, 2018

umomany commented May 8, 2019

umomany commented May 8, 2019

bacongobbler commented May 8, 2019

YingjunHu commented Jul 19, 2019

bacongobbler commented Aug 14, 2019

YingjunHu commented Aug 14, 2019

bacongobbler commented Aug 14, 2019

bacongobbler commented Aug 19, 2020 • edited

bacongobbler commented Aug 19, 2020

github-actions bot commented Nov 22, 2020

Bessonov commented Nov 22, 2020

bacongobbler commented Dec 16, 2020

bacongobbler commented Dec 16, 2020 • edited

Bessonov commented Dec 16, 2020

jerriais commented Jan 19, 2021

bacongobbler commented Jan 19, 2021 • edited

archerbj commented Mar 9, 2021

Ornias1993 commented Mar 13, 2021 • edited

ishaan-rd commented May 8, 2021

eranelbaz commented Jul 11, 2021

patsevanton commented Jul 13, 2021 • edited

bacongobbler commented Aug 19, 2020 •

edited

bacongobbler commented Dec 16, 2020 •

edited

bacongobbler commented Jan 19, 2021 •

edited

Ornias1993 commented Mar 13, 2021 •

edited

patsevanton commented Jul 13, 2021 •

edited