Labeller pods are not created

[itzikb-redhat]

After running the attached deploy.sh script there are no node labeller nodes:

$ oc get pods
NAME                                              READY   STATUS    RESTARTS   AGE
cdi-apiserver-799b86cd47-blq9l                    1/1     Running   0          3h55m
cdi-deployment-67855b764d-lhc47                   1/1     Running   0          3h55m
cdi-operator-86cfbc4f55-hzv98                     1/1     Running   0          3h55m
cdi-uploadproxy-7cd5bdb789-9jlfr                  1/1     Running   0          3h55m
cluster-network-addons-operator-c78d4f6fc-m86r7   1/1     Running   0          3h55m
hyperconverged-cluster-operator-d6c986564-p6lqd   1/1     Running   0          3h55m
kubevirt-ssp-operator-c4cd5f564-lp8mj             1/1     Running   0          3h55m
kubevirt-web-ui-operator-59f778956-jkvlr          1/1     Running   0          3h55m
node-maintenance-operator-6b6d5756d8-gbnw7        1/1     Running   0          3h55m
virt-operator-8fd6c4c4c-hj6x2                     1/1     Running   0          3h55m
virt-operator-8fd6c4c4c-ts58n                     1/1     Running   0          3h55m

$ virtctl version
Client Version: version.Info{GitVersion:"v0.17.4", GitCommit:"adfdb8c07830b99fc79d2fd1d004e862ef70979e", GitTreeState:"clean", BuildDate:"2019-06-27T17:03:10Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

[rthallisey]

With the pods listed here, it looks like the HCO's CR was never created. Can you check oc get hco -n kubevirt-hyperconverged? If it exists, can you paste the logs from the HCO pod?

[itzikb-redhat]

Here are the last lines from HCO log

[rthallisey]

The logs are showing that all the component CRs exist. Can you check if oc get kubevirt -n kubevirt-hyperconverged exists? Also, I'm wondering if #104 will fix this. Can you re deploy to pick up the latest build?

[itzikb-redhat]

Yes.
$ oc get kubevirt -n kubevirt-hyperconverged
NAME AGE PHASE
kubevirt-hyperconverged-cluster 10h Deploying

I'll redeploy.

[gklein]

Still failing to deploy the virt-labelers with the latest HCO build:

hyperconverged-cluster-operator$ git rev-parse HEAD
53df83bd125ab45f1a83d5d2492b24bc4f11d783

$ oc get pods -n kubevirt-hyperconverged
NAME                                              READY   STATUS    RESTARTS   AGE
cdi-apiserver-799b86cd47-55mcd                    1/1     Running   0          5m49s
cdi-deployment-67855b764d-925sd                   1/1     Running   0          5m48s
cdi-operator-86cfbc4f55-6dq2v                     1/1     Running   0          6m17s
cdi-uploadproxy-7cd5bdb789-jrn4j                  1/1     Running   0          5m49s
cluster-network-addons-operator-c78d4f6fc-jbx89   1/1     Running   0          6m17s
hyperconverged-cluster-operator-d6c986564-79qjg   1/1     Running   0          6m18s
kubevirt-ssp-operator-c4cd5f564-tcbbm             1/1     Running   0          6m17s
kubevirt-web-ui-operator-59f778956-l2mv4          1/1     Running   0          6m18s
node-maintenance-operator-6b6d5756d8-mgzsh        1/1     Running   0          6m18s
virt-operator-8fd6c4c4c-28mh4                     1/1     Running   0          6m18s
virt-operator-8fd6c4c4c-548vv                     1/1     Running   0          6m18s

$ oc get hco -n kubevirt-hyperconverged
NAME                     AGE     PHASE
hyperconverged-cluster   7m40s

$ oc get kubevirt -n kubevirt-hyperconverged
NAME                              AGE     PHASE
kubevirt-hyperconverged-cluster   7m32s   Deploying

[gklein]

Some more logs from kubevirt-ssp-operator

$ oc logs kubevirt-ssp-operator-c4cd5f564-tcbbm
E0702 10:46:54.644438       1 reflector.go:134] sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:126: Failed to list *unstructured.Unstructured: kubevirttemplatevalidators.kubevirt.io is forbidden: User "system:serviceaccount:kubevirt-hyperconverged:kubevirt-ssp-operator" cannot list resource "kubevirttemplatevalidators" in API group "kubevirt.io" at the cluster scope
E0702 10:46:55.640944       1 reflector.go:134] sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:126: Failed to list *unstructured.Unstructured: kubevirtnodelabellerbundles.kubevirt.io is forbidden: User "system:serviceaccount:kubevirt-hyperconverged:kubevirt-ssp-operator" cannot list resource "kubevirtnodelabellerbundles" in API group "kubevirt.io" at the cluster scope
E0702 10:46:55.642286       1 reflector.go:134] sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:126: Failed to list *unstructured.Unstructured: kubevirtcommontemplatesbundles.kubevirt.io is forbidden: User "system:serviceaccount:kubevirt-hyperconverged:kubevirt-ssp-operator" cannot list resource "kubevirtcommontemplatesbundles" in API group "kubevirt.io" at the cluster scope
E0702 10:46:55.646566       1 reflector.go:134] sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:126: Failed to list *unstructured.Unstructured: kubevirttemplatevalidators.kubevirt.io is forbidden: User "system:serviceaccount:kubevirt-hyperconverged:kubevirt-ssp-operator" cannot list resource "kubevirttemplatevalidators" in API group "kubevirt.io" at the cluster scope