Support extended resource in NodeResourcesBalancedAllocation plugin #101946

chendave · 2021-05-12T10:55:50Z

Per the design https://docs.google.com/document/d/1MT0KVnyeAc4QcKqUsmk-4OvmBhOy6PvBQOGnjugrJ9k/edit?ts=60942305# and the discussion in the scheduler meeting.

Signed-off-by: Dave Chen dave.chen@arm.com

What type of PR is this?

/kind feature

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Scheduler could be configured  to consider new resources beside CPU and memory,  GPU for example, for the score plugin of `NodeResourcesBalancedAllocation`.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot · 2021-05-12T10:55:57Z

@chendave: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

fejta-bot · 2021-05-12T11:24:35Z

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

pkg/scheduler/framework/plugins/noderesources/balanced_allocation.go

chendave · 2021-05-13T06:22:52Z

This is ready for review.

cc more maintainers in case any of you have some bandwidth to review this change, thanks in the advance!

@Huang-Wei @ahg-g @alculquicondor

alculquicondor · 2021-05-13T13:34:24Z

The release note is not accurate. It kind of implies that, by default, kube-scheduler will start considering new resources. It should say that it can be configured to consider new resources.

pkg/scheduler/apis/config/types_pluginargs.go

pkg/scheduler/apis/config/v1beta1/defaults.go

pkg/scheduler/framework/plugins/noderesources/balanced_allocation.go

pkg/scheduler/apis/config/types_pluginargs.go

pkg/scheduler/framework/plugins/noderesources/balanced_allocation.go

pkg/scheduler/apis/config/v1beta1/defaults.go

pkg/scheduler/apis/config/validation/validation.go

pkg/scheduler/apis/config/validation/validation_pluginargs.go

pkg/scheduler/framework/plugins/noderesources/balanced_allocation.go

pkg/scheduler/framework/plugins/noderesources/resource_allocation.go

staging/src/k8s.io/kube-scheduler/config/v1beta1/register.go

pkg/scheduler/apis/config/v1beta1/defaults_test.go

pkg/scheduler/apis/config/validation/validation_pluginargs_test.go

Huang-Wei

Can you retitle to "Support extended resource in NodeResourcesBalancedAllocation plugin"?

chendave · 2021-05-14T10:01:54Z

/retitle: Support extended resource in NodeResourcesBalancedAllocation plugin

chendave · 2021-05-14T10:05:05Z

Thanks guys for your great suggestion!

Couple of them are pending on discussion, will update soon when it is get clear.

pkg/scheduler/apis/config/types_pluginargs.go

pkg/scheduler/apis/config/v1beta1/defaults.go

staging/src/k8s.io/kube-scheduler/config/v1beta1/types_pluginargs.go

pkg/scheduler/apis/config/validation/validation_pluginargs.go

pkg/scheduler/framework/plugins/noderesources/balanced_allocation.go

chendave · 2021-05-19T10:48:45Z

/hold

TODO

~~update testcase~~
~~rebase~~

pkg/scheduler/apis/config/types_pluginargs.go

pkg/scheduler/framework/plugins/noderesources/balanced_allocation.go

staging/src/k8s.io/kube-scheduler/config/v1beta1/types_pluginargs.go

pkg/scheduler/framework/plugins/noderesources/balanced_allocation.go

pkg/scheduler/framework/plugins/noderesources/balanced_allocation_test.go

Huang-Wei · 2021-06-28T20:00:41Z

@Huang-Wei, this is compatible with your proposal in #103169, right?

@alculquicondor it's compatible and UTed properly.

chendave

Thanks guys for the review and those great suggestion, I hope all those comments have been addressed.

pkg/scheduler/apis/config/validation/validation_pluginargs.go

pkg/scheduler/framework/plugins/noderesources/balanced_allocation.go

pkg/scheduler/framework/plugins/noderesources/balanced_allocation_test.go

staging/src/k8s.io/kube-scheduler/config/v1beta1/types_pluginargs.go

pkg/scheduler/apis/config/scheme/scheme_test.go

alculquicondor · 2021-06-29T13:59:01Z

pkg/scheduler/apis/config/scheme/scheme_test.go

@@ -381,6 +381,82 @@ profiles:
 				},
 			},
 		},
+		{
+			name: "v1beta1 NodeResourcesBalancedAllocation resource weights are set properly",


This is kind of too much, I would just reuse one of the existing "custom configs" test cases.
As it is, it will add toil for the maintenance of this file.

squeezed this test into "...all plugin args in default profile".

…ources Signed-off-by: Dave Chen <dave.chen@arm.com>

chendave · 2021-06-30T03:17:51Z

squashed the commits since it will easier for me to rebase on the head of master.

chendave · 2021-07-01T10:34:10Z

@alculquicondor any more comments, or can we get this in 1.22? thanks!

Huang-Wei · 2021-07-01T18:29:13Z

/lgtm

Leaving /approve to @alculquicondor .

alculquicondor

/approve

Sorry, I was out on vacation.

alculquicondor · 2021-07-05T13:43:34Z

/label api-review

chendave · 2021-07-05T14:12:19Z

thanks @alculquicondor , hope you've had a good vacation!

I guess this needs the approval from api review, @liggitt would you pls take the final review?

chendave · 2021-07-05T14:13:00Z

/cc @liggitt for api review, thanks!

liggitt · 2021-07-06T15:54:58Z

/approve

k8s-ci-robot · 2021-07-06T15:55:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alculquicondor, chendave, liggitt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~cmd/kube-scheduler/OWNERS~~ [alculquicondor,liggitt]
~~pkg/scheduler/OWNERS~~ [alculquicondor,liggitt]
~~staging/src/k8s.io/kube-scheduler/config/OWNERS~~ [liggitt]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label May 12, 2021

k8s-ci-robot requested review from damemi and hex108 May 12, 2021 10:56

chendave force-pushed the balance_allocation branch from 69222d8 to c0bc925 Compare May 12, 2021 10:56

chendave force-pushed the balance_allocation branch from c0bc925 to 3b73c18 Compare May 13, 2021 05:54

chendave commented May 13, 2021

View reviewed changes

pkg/scheduler/framework/plugins/noderesources/balanced_allocation.go Show resolved Hide resolved

alculquicondor reviewed May 13, 2021

View reviewed changes

Huang-Wei reviewed May 13, 2021

View reviewed changes

k8s-ci-robot changed the title ~~Extent NodeResourcesBalancedAllocation plugin to cover more resources~~ : Support extended resource in NodeResourcesBalancedAllocation plugin May 14, 2021

chendave changed the title ~~: Support extended resource in NodeResourcesBalancedAllocation plugin~~ Support extended resource in NodeResourcesBalancedAllocation plugin May 14, 2021

alculquicondor reviewed May 14, 2021

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 14, 2021

Huang-Wei mentioned this pull request May 18, 2021

Add score func for NodeResourcesFit plugin #101822

Merged

ahg-g reviewed Jun 28, 2021

View reviewed changes

Huang-Wei reviewed Jun 28, 2021

View reviewed changes

chendave commented Jun 29, 2021

View reviewed changes

chendave force-pushed the balance_allocation branch 2 times, most recently from 137d21d to 2bfa67f Compare June 29, 2021 12:38

alculquicondor reviewed Jun 29, 2021

View reviewed changes

Extent the NodeResourcesBalancedAllocation plugin to cover more res…

1fa673c

…ources Signed-off-by: Dave Chen <dave.chen@arm.com>

chendave force-pushed the balance_allocation branch from 2bfa67f to 1fa673c Compare June 30, 2021 03:15

k8s-ci-robot assigned Huang-Wei Jul 1, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 1, 2021

alculquicondor reviewed Jul 5, 2021

View reviewed changes

k8s-ci-robot added the api-review Categorizes an issue or PR as actively needing an API review. label Jul 5, 2021

liggitt added this to API review completed, 1.22 in API Reviews Jul 6, 2021

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 6, 2021

k8s-ci-robot merged commit ea3bcbc into kubernetes:master Jul 6, 2021

k8s-ci-robot added this to the v1.22 milestone Jul 6, 2021

chendave deleted the balance_allocation branch July 11, 2021 03:35

This was referenced Sep 21, 2021

UPSTREAM: revert 102925: Revert calculation for resource scoring openshift/kubernetes#972

Closed

UPSTREAM: revert 102925: Revert calculation for resource scoring openshift/kubernetes#973

Closed

alculquicondor mentioned this pull request Sep 24, 2021

Pods not being evenly scheduled across worker nodes #105220

Closed

chendave mentioned this pull request Feb 21, 2022

2021 Annual Report: SIG Scheduling kubernetes/community#6467

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support extended resource in NodeResourcesBalancedAllocation plugin #101946

Support extended resource in NodeResourcesBalancedAllocation plugin #101946

chendave commented May 12, 2021 •

edited

Loading

k8s-ci-robot commented May 12, 2021

fejta-bot commented May 12, 2021

chendave commented May 13, 2021

alculquicondor commented May 13, 2021

Huang-Wei left a comment

chendave commented May 14, 2021

chendave commented May 14, 2021

chendave commented May 19, 2021 •

edited

Loading

Huang-Wei commented Jun 28, 2021

chendave left a comment

alculquicondor Jun 29, 2021

chendave Jun 30, 2021

chendave commented Jun 30, 2021

chendave commented Jul 1, 2021

Huang-Wei commented Jul 1, 2021

alculquicondor left a comment

alculquicondor commented Jul 5, 2021

chendave commented Jul 5, 2021

chendave commented Jul 5, 2021

liggitt commented Jul 6, 2021

k8s-ci-robot commented Jul 6, 2021

Support extended resource in NodeResourcesBalancedAllocation plugin #101946

Support extended resource in NodeResourcesBalancedAllocation plugin #101946

Conversation

chendave commented May 12, 2021 • edited Loading

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot commented May 12, 2021

fejta-bot commented May 12, 2021

chendave commented May 13, 2021

alculquicondor commented May 13, 2021

Huang-Wei left a comment

Choose a reason for hiding this comment

chendave commented May 14, 2021

chendave commented May 14, 2021

chendave commented May 19, 2021 • edited Loading

Huang-Wei commented Jun 28, 2021

chendave left a comment

Choose a reason for hiding this comment

alculquicondor Jun 29, 2021

Choose a reason for hiding this comment

chendave Jun 30, 2021

Choose a reason for hiding this comment

chendave commented Jun 30, 2021

chendave commented Jul 1, 2021

Huang-Wei commented Jul 1, 2021

alculquicondor left a comment

Choose a reason for hiding this comment

alculquicondor commented Jul 5, 2021

chendave commented Jul 5, 2021

chendave commented Jul 5, 2021

liggitt commented Jul 6, 2021

k8s-ci-robot commented Jul 6, 2021

chendave commented May 12, 2021 •

edited

Loading

chendave commented May 19, 2021 •

edited

Loading