Upgrading from Gloe EE 1.6.31 to 1.7.12 fails

[willowmck]

Describe the bug
Upgrading via helm with no values results in the following error =>
Error: UPGRADE FAILED: cannot patch "glooe-prometheus-kube-state-metrics" with kind Service: Service "glooe-prometheus-kube-state-metrics" is invalid: spec.clusterIPs[0]: Invalid value: []string(nil): primary clusterIP can not be unset && cannot patch "glooe-prometheus-kube-state-metrics" with kind Deployment: Deployment.apps "glooe-prometheus-kube-state-metrics" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/name":"kube-state-metrics"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

To Reproduce
Steps to reproduce the behavior:

1. Install Gloo Edge Enterprise v1.6.31 with Helm
2. Upgrade to Gloo Edge Enterprise v1.7.12 with Helm
3. See error

Expected behavior
Upgrade should succeed.

Additional context
N/A

[chrisgaun]

@sam-heilbron put an explanation of workaround and close out.

[ably77]

I worked around this by disabling the default prometheus install in a helm values file called values-override.yaml like this:

prometheus:
  enabled: false
<omitted other existing helm values>

then run upgrades

helm upgrade gloo glooe/gloo-ee --namespace gloo-system --version $VERSION --set license_key=$LICENSE_KEY --values=values-override.yaml

After upgrading, you can then re-enable prometheus which will redeploy the chart