Upgrading the Kubeapps chart deletes the apprepositores CRD

[andresmgot]

Somehow the upgrade process is deleting the apprepositories CRD for me:

▶ helm install --name kubeapps  --namespace kubeapps bitnami/kubeapps
NAME:   kubeapps
LAST DEPLOYED: Fri Nov  9 13:18:13 2018
NAMESPACE: kubeapps
STATUS: DEPLOYED
...
▶ kubectl get crd
NAME                           CREATED AT
apprepositories.kubeapps.com   2018-11-09T12:14:36Z
▶ helm upgrade --namespace kubeapps kubeapps bitnami/kubeapps
Release "kubeapps" has been upgraded. Happy Helming!
LAST DEPLOYED: Fri Nov  9 13:18:47 2018
NAMESPACE: kubeapps
STATUS: DEPLOYED
...
▶ kubectl get crd
No resources found.

[andresmgot]

I am already investigating the issue

[arapulido]

Thanks @andresmgot for the heads up. Let us know

[andresmgot]

Fix is available in #801 but I am not sure why I am seeing this now.

[prydonius]

I think the reason why we haven't always noticed this is because you will only run into this if you try to upgrade the first Kubeapps release you installed in your cluster. If you install Kubeapps for the first time, and then never upgrade but delete the release, the App CRD will be kept due to the resource policy annotation. Subsequent installs of the Kubeapps chart will not include the App CRD (because it already exists), and so upgrading does not have this issue.

[migmartri]

I can confirm @prydonius theory.

Basically the system behaves correctly if you have installed and deleted a previous release, meaning that you are upgrading another one.

Scenario 1: You are upgrading the first instance of Kubeapps ever installed in the cluster

- install kubeapps
- upgrade kubeapps # the crd will be gone

Scenario 2: You are upgrading a second+ instance of Kubeapps

- install kubeapps
- delete kubeapps
- install kubeapps again
- upgrade kubeapps # the crd is kept

I can confirm that in my case I am experiencing the scenario 2.

I am running kubeapps 0.9.4

$ helm list miguel-kubeapps
NAME            REVISION        UPDATED                         STATUS          CHART           NAMESPACE
miguel-kubeapps 42              Fri Nov  2 17:18:04 2018        DEPLOYED        kubeapps-0.9.4  miguel   

which manifest does not include the CRD

$ helm get manifest miguel-kubeapps | grep CustomResourceDefinition
# nothing

but I have a crd installed by Kubeapps 0.3.0

$ kubectl get crd apprepositories.kubeapps.com -o yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    helm.sh/resource-policy: keep
  creationTimestamp: 2018-08-27T21:08:03Z
  generation: 1
  labels:
    app: miguel-kubeapps-apprepository-controller
    chart: kubeapps-0.3.0
    heritage: Tiller
    release: miguel-kubeapps

version that is not part of my current release, note that my first one is 0.4.6

$ helm history miguel-kubeapps
REVISION        UPDATED                         STATUS          CHART           DESCRIPTION                                                 
1               Wed Sep 12 17:31:17 2018        SUPERSEDED      kubeapps-0.4.6  Install complete                                            
2               Wed Sep 19 17:04:59 2018        FAILED          kubeapps-0.4.7  Upgrade "miguel-kubeapps" failed: Deployment.apps "miguel...
3               Wed Sep 19 17:05:48 2018        SUPERSEDED      kubeapps-0.4.6  Rollback to 1                                               
4               Wed Sep 19 17:06:32 2018        SUPERSEDED      kubeapps-0.4.8  Upgrade complete                                            
5               Thu Sep 20 15:16:59 2018        SUPERSEDED      kubeapps-0.5.0  Upgrade complete                                            
6               Fri Sep 21 11:47:51 2018        SUPERSEDED      kubeapps-0.5.1  Upgrade complete                                            
7               Fri Sep 21 12:01:44 2018        SUPERSEDED      kubeapps-0.5.1  Upgrade complete        

[migmartri]

BTW, I think that this is an instance of the issue #625