Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure the cluster autoscaler has the highest pod priority #2346

Closed
RobertLucian opened this issue Jul 15, 2021 · 0 comments · Fixed by #2347
Closed

Ensure the cluster autoscaler has the highest pod priority #2346

RobertLucian opened this issue Jul 15, 2021 · 0 comments · Fixed by #2347
Assignees
Labels
provisioning Something related to cluster provisioning
Milestone

Comments

@RobertLucian
Copy link
Member

Description

Required to ensure that if a pod gets evicted due to low resource availability on the node, the cluster autoscaler can provision another node for the operator node group. When the node's kubelet notices the low resource availability (memory), it taints the node with the node.kubernetes.io/memory-pressure:NoSchedule taint, which will effectively trigger the autoscaler to add another node for the other pending pods.

Whereas if it doesn't have the highest priority, there's a chance of the cluster autoscaler pod to get evicted, which will leave the cluster in a permanently broken state - it is possible that most of the pods won't be able to start on the nodes of the operator node group because they will all be demanding more memory because:

a). They would all use more memory on startup than usual (given the node's already dwindling resources)
b). The pods may be requiring more memory due to the cluster's big size.

@RobertLucian RobertLucian added bug Something isn't working performance A performance improvement labels Jul 15, 2021
@RobertLucian RobertLucian self-assigned this Jul 15, 2021
@miguelvr miguelvr added provisioning Something related to cluster provisioning and removed bug Something isn't working performance A performance improvement labels Jul 16, 2021
@deliahu deliahu added this to the v0.39 milestone Jul 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
provisioning Something related to cluster provisioning
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants