Update the make helmfile/destroy/all command for helm3

[willgraf]

helm3 updates the helm delete command to require the namespace of the helm deployment, which breaks our current implementation. Fortunately, helmfile destroy will delete all installed helm charts regardless of namespace, and is a much simpler command than piping each name/namespace to helm destroy.

This issue was not discovered during the helm 3 migration as the make command only deleted the deepcell namespace, the other pods were destroyed during cluster destruction and did not have any PVCs that got stranded.

[willgraf]

This PR is blocked by a prometheus-operator==8.12.3 issue when deleting CRDs installed by prometheus operator. This should be resolved by #317.

[willgraf]

The issue is NOT resolved. Despite prometheus-operator being upgraded, helmfile destroy does NOT successfully delete it:

Error: uninstallation completed with 1 error(s): unable to build kubernetes objects for delete: [unable to recognize "": no matches for kind "Alertmanager" in version "monitoring.coreos.com/v1", unable to recognize no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "Prometheus" in version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"]

I believe a manual workaround involves turning off cleanupCustomResource and manually deleting the CRDs.

---

There is more work required in figuring out how to reliably delete the helm charts during cluster tear-down.

[willgraf]

This issue is being worked on in another PR #406