Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Activate unschedulable pods only if the node became more schedulable #70366

Merged
merged 1 commit into from Oct 30, 2018
Merged

Activate unschedulable pods only if the node became more schedulable #70366

merged 1 commit into from Oct 30, 2018

Conversation

mlmhl
Copy link
Contributor

@mlmhl mlmhl commented Oct 29, 2018 •
edited by bsalamat

What type of PR is this?
/kind feature

What this PR does / why we need it:

This is a performance optimization for scheduler:

Move unschedulable pods to active queue only if a node's scheduling related properties updated. This PR considers node allocatable, node conditions, node taints, node labels and Node.Spec.Unschedulable as scheduling related properties.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #70316

Does this PR introduce a user-facing change?:

Scheduler only activates unschedulable pods if node's scheduling related properties change.

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Oct 29, 2018
@wgliang
Copy link
Contributor

wgliang commented Oct 29, 2018 •
edited

@mlmhl please make the CI happy(seem like you need to gofmt the code).

@wgliang
Copy link
Contributor

wgliang commented Oct 29, 2018

And I think this PR need a effective release note.

bsalamat
bsalamat reviewed Oct 29, 2018
View reviewed changes
Copy link
Member

@bsalamat bsalamat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for working on this so quickly. Given that nodes send updates every 10 seconds, in large clusters we receive hundreds of node updates per second. So, it is important that the checks be very quick. That's the main reason that I suggested a couple of changes to make the logic simpler. Of course, simpler logic is easier to maintain as well.

pkg/scheduler/factory/factory.go Outdated
if reflect.DeepEqual(oldAllocatable, newAllocatable) {
return false
}
for resource, newValue := range newAllocatable {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What you have done makes sense, but in order to be quicker in checking node changes and also to be conservative, I would return "true" as long allocatables are changed, no matter whether they are reduced or increased. In other words, remove this for loop and just return true.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

pkg/scheduler/factory/factory.go Outdated
return false
}

healthyConditions := []v1.NodeConditionType{v1.NodeReady}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to my previous comment, I would return true as long as old and new conditions are not equal. This will help reduce chances of introducing bugs in the future when new node conditions are added.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Oct 30, 2018
@mlmhl
Copy link
Contributor Author

mlmhl commented Oct 30, 2018

@bsalamat @wgliang All comments are updated, PTAL :)

bsalamat
bsalamat approved these changes Oct 30, 2018
View reviewed changes
Copy link
Member

@bsalamat bsalamat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

Thanks, @mlmhl!

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 30, 2018
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bsalamat, mlmhl

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 30, 2018
@mlmhl
Copy link
Contributor Author

mlmhl commented Oct 30, 2018

/test pull-kubernetes-e2e-gce-100-performance

@k8s-ci-robot k8s-ci-robot merged commit fda41d1 into kubernetes:master Oct 30, 2018
@wojtek-t wojtek-t mentioned this pull request Nov 7, 2018
Closed
@bsalamat bsalamat mentioned this pull request Nov 8, 2018
Merged
@bsalamat bsalamat mentioned this pull request Nov 28, 2018
Closed
@mlmhl mlmhl deleted the scheduler_optimization branch November 29, 2018 07:23
@k82cn k82cn mentioned this pull request Nov 30, 2018
Merged
@xiaoxubeii xiaoxubeii mentioned this pull request Dec 5, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bsalamat bsalamat bsalamat approved these changes

@aveshagarwal aveshagarwal Awaiting requested review from aveshagarwal

Assignees

@bsalamat bsalamat

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Process unschedulable pods on node updates more efficiently
4 participants
@mlmhl @wgliang @k8s-ci-robot @bsalamat