switch to DaemonSet to run on all control plane nodes #310
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Partially addresses #304, specifically the second issue raised.
Rather than running it as a
Deployment
withreplicas=1
, we run this as aDaemonSet
withnodeAffinity
restricting it to control plane nodes. This means that if you have 3 such nodes, you will get 3 copies.The built-in leader election of k8s.io/cloud-provider ensures that only one processes events at a time.
If you have multiple (as this provides), and one dies (of greater concern, the node it is running on dies, taking CPEM with it), then one of the remaining ones will take over quickly, and work will continue as normal. This includes the important part of letting the apiserver know that the node is gone, which is a CPEM responsibility.
Important note:: This does not solve the issue of the node that dies being host both to the CPEM (whether lone before this PR or current leader after this PR) and the EIP (when using an EIP managed by CPEM for apiserver access. In that case, this will not help. As the node goes down, so does CPEM, so nothing can switch the EIP to a functioning node. That is CPEM's responsibility, but it is down, too. Leader election would help, but leader election depends on access to the k8s apiserver, which depends on the EIP, which points to the node that just went down.
That will need to be addressed via some other solution; see the tracking issue linked at the beginning.
Separately, while also dealing with the deployment templates, it also fixes a deprecated annotation. This used to exist:
But has been deprecated since 1.16. The replacement to use:
as part of the podspec has been adopted.