Server Forbidden Updates To This Resource

[dgoodwin]

Not sure what I've got here but this is a CI failure bringing up a cluster, from the CVO pod logs:

E0213 17:40:08.957050       1 task.go:57] error running apply for service "openshift-cloud-credential-operator/controller-manager-service" (84 of 273): services "controller-manager-service" is forbidden: caches not synchronized
E0213 17:40:28.980158       1 task.go:57] error running apply for service "openshift-cloud-credential-operator/controller-manager-service" (84 of 273): services "controller-manager-service" is forbidden: caches not synchronized
I0213 17:40:38.501814       1 leaderelection.go:209] successfully renewed lease openshift-cluster-version/version
I0213 17:40:40.483182       1 reflector.go:286] github.com/openshift/cluster-version-operator/vendor/github.com/openshift/client-go/config/informers/externalversions/factory.go:101: forcing resync
E0213 17:40:51.996136       1 task.go:57] error running apply for service "openshift-cloud-credential-operator/controller-manager-service" (84 of 273): services "controller-manager-service" is forbidden: caches not synchronized
I0213 17:40:51.996197       1 task_graph.go:438] No more reachable nodes in graph, continue
I0213 17:40:51.996203       1 task_graph.go:474] No more work
I0213 17:40:51.996221       1 task_graph.go:494] No more work for 3
I0213 17:40:51.996227       1 task_graph.go:494] No more work for 6
I0213 17:40:51.996234       1 task_graph.go:494] No more work for 7
I0213 17:40:51.996240       1 task_graph.go:494] No more work for 1
I0213 17:40:51.996246       1 task_graph.go:494] No more work for 4
I0213 17:40:51.996252       1 task_graph.go:494] No more work for 2
I0213 17:40:51.996252       1 task_graph.go:494] No more work for 0
I0213 17:40:51.996257       1 task_graph.go:494] No more work for 5
I0213 17:40:51.996277       1 task_graph.go:510] Workers finished
I0213 17:40:51.996290       1 task_graph.go:518] Result of work: [Could not update service "openshift-cloud-credential-operator/controller-manager-service" (84 of 273): the server has forbidden updates to this resource]
E0213 17:40:51.996341       1 sync_worker.go:263] unable to synchronize image (waiting 3m19.747206386s): Could not update service "openshift-cloud-credential-operator/controller-manager-service" (84 of 273): the server has forbidden updates to this resource
I0213 17:40:51.996400       1 cvo.go:298] Started syncing cluster version "openshift-cluster-version/version" (2019-02-13 17:40:51.996393402 +0000 UTC m=+2487.354191867)
I0213 17:40:51.996446       1 cvo.go:326] Desired version from operator is v1.Update{Version:"0.0.1-2019-02-13-164905", Image:"registry.svc.ci.openshift.org/ci-op-girsxxlp/release@sha256:ded54f5fb7dfe10f53176ac710f6309b05828dc0aa276b448ce5aefc8e5eae78"}
I0213 17:40:51.996541       1 cvo.go:300] Finished syncing cluster version "openshift-cluster-version/version" (144.1µs)

More logs available here: https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/openshift_cloud-credential-operator/31/pull-ci-openshift-cloud-credential-operator-master-e2e-aws/158

[dgoodwin]

Some additional analysis in openshift/cloud-credential-operator#31 (comment)

I think it's my PR but I don't understand why.

[dgoodwin]

A little more info here: openshift/cloud-credential-operator#31 (comment)

[abhinavdahiya]

openshift/cloud-credential-operator#31 (comment)

This was not a CVO bug, operators running before openshift-apiserver-operator need to special case their namespaces so the kube-apiserver can accept resources their namespaces in absence of caches synced for the all the admission plugins provided by the openshift-apiserver.

Special label:

cluster-version-operator/install/0000_00_cluster-version-operator_00_namespace.yaml

Line 7 in 93d4eff

openshift.io/run-level: "1"

/close

[openshift-ci-robot]

@abhinavdahiya: Closing this issue.

In response to this:

openshift/cloud-credential-operator#31 (comment)

This was not a CVO bug, operators running before openshift-apiserver-operator need to special case their namespaces so the kube-apiserver can accept resources their namespaces in absence of caches synced for the all the admission plugins provided by the openshift-apiserver.

Special label:

cluster-version-operator/install/0000_00_cluster-version-operator_00_namespace.yaml

Line 7 in 93d4eff

openshift.io/run-level: "1"

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.