Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkg/k8s: use a controller manager in K8sClient #17112

Merged
merged 2 commits into from Aug 23, 2021

Conversation

aanm
Copy link
Member

@aanm aanm commented Aug 9, 2021

Since "controller.NewManager()" returns a new Manager, triggering a
controller from this newly returned manager does not work because the
manager does not have any controller set in it. In order to be able
to re-trigger controllers, a single instance of this manager is set in
the K8sClient struct.

Fixes: bd34b95 ("pkg/k8s: remove node.cilium.io/agent-not-ready taint from nodes")
Signed-off-by: André Martins andre@cilium.io

Fixes: #17103

Remove `node.cilium.io/agent-not-ready` node taints if they are re-added after Cilium has started

@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Aug 9, 2021
@aanm aanm added kind/bug This is a bug in the Cilium logic. release-note/bug This PR fixes an issue in a previous release of Cilium. sig/k8s Impacts the kubernetes API, or kubernetes -> cilium internals translation layers. labels Aug 9, 2021
@maintainer-s-little-helper maintainer-s-little-helper bot removed dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. labels Aug 9, 2021
@aanm aanm force-pushed the pr/fix-mark-k8s-node-ready branch from 8a2c4db to 821a2a3 Compare August 10, 2021 00:35
@aanm aanm requested a review from gandro August 10, 2021 00:37
@aanm
Copy link
Member Author

aanm commented Aug 10, 2021

test-me-please

@aanm aanm marked this pull request as ready for review August 10, 2021 02:02
@aanm aanm requested review from a team, joamaki and nathanjsweet August 10, 2021 02:02
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from master in 1.10.4 Aug 10, 2021
@@ -282,6 +282,7 @@ func (c *Controller) runController() {

case <-runTimer.After(interval):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it on purpose that we do nothing after the timer elapses? Go doesn't do case fallthrough.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indentation seems to be a bit off as well :-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it on purpose that we do nothing after the timer elapses? Go doesn't do case fallthrough.

We do "something". Once that timer elapses we run the function in

err = params.DoFunc(c.ctxDoFunc)

Indentation seems to be a bit off as well :-)

The indentation is correct, otherwise gofmt would have complained.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah now I get the logic. And I guess it's just github not showing it properly.

Since "controller.NewManager()" returns a new Manager, triggering a
controller from this newly returned manager does not work because the
manager does not have any controller set in it. In order to be able
to re-trigger controllers, a single instance of this manager is set in
the K8sClient struct.

Fixes: bd34b95 ("pkg/k8s: remove node.cilium.io/agent-not-ready taint from nodes")
Signed-off-by: André Martins <andre@cilium.io>
`runFunc` should be set to true when the controller is manually
triggered otherwise the controller might never be executed in the case
where its run interval is zero.

Fixes: c61d02f ("controller: allow to manually trigger it")
Signed-off-by: André Martins <andre@cilium.io>
@aanm aanm force-pushed the pr/fix-mark-k8s-node-ready branch from 821a2a3 to 91b8de5 Compare August 18, 2021 14:57
@aanm
Copy link
Member Author

aanm commented Aug 18, 2021

test-me-please

k8s-1.20-kernel-4.19 failed with:

Aug 18 15:51:43 k8s2 kubelet[4826]: E0818 15:51:43.665082    4826 kube_docker_client.go:340] Cancel pulling image "quay.io/cilium/istio_pilot:1.10.3" because of no progress for 1m0s, latest progress: "25fa05cd42bd: Downloading [================>                                  ]  8.895MB/26.7MB"
Aug 18 15:51:43 k8s2 kubelet[4826]: E0818 15:51:43.666001    4826 remote_image.go:113] PullImage "quay.io/cilium/istio_pilot:1.10.3" from image service failed: rpc error: code = Unknown desc = context canceled
Aug 18 15:51:43 k8s2 kubelet[4826]: E0818 15:51:43.666112    4826 kuberuntime_image.go:51] Pull image "quay.io/cilium/istio_pilot:1.10.3" failed: rpc error: code = Unknown desc = context canceled

gke-stable was unable to be provisioned https://jenkins.cilium.io/job/Cilium-PR-K8s-GKE/6245/flowGraphTable

@aanm
Copy link
Member Author

aanm commented Aug 20, 2021

test-gke

@aanm
Copy link
Member Author

aanm commented Aug 20, 2021

test-1.20-4.19

@aanm
Copy link
Member Author

aanm commented Aug 23, 2021

ci-gke

@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Aug 23, 2021
@aanm aanm merged commit 4ebcc27 into cilium:master Aug 23, 2021
@aanm aanm deleted the pr/fix-mark-k8s-node-ready branch August 23, 2021 11:39
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from master to Backport pending to v1.10 in 1.10.4 Aug 30, 2021
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.10 to Backport done to v1.10 in 1.10.4 Sep 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug This is a bug in the Cilium logic. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/bug This PR fixes an issue in a previous release of Cilium. sig/k8s Impacts the kubernetes API, or kubernetes -> cilium internals translation layers.
Projects
No open projects
1.10.4
Backport done to v1.10
Development

Successfully merging this pull request may close these issues.

Cilium does not remove taint on already running nodes
6 participants