Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Scheduler validation check for redeclared policy configs #83963

Merged
merged 2 commits into from Oct 30, 2019
Merged

Add Scheduler validation check for redeclared policy configs #83963

merged 2 commits into from Oct 30, 2019

Conversation

damemi
Copy link
Contributor

@damemi damemi commented Oct 15, 2019

What type of PR is this?
/kind bug

What this PR does / why we need it:
The ability to re-declare custom policy configs was not entirely intentional, and this adds validation to disallow that in the new framework

Which issue(s) this PR fixes:
Fixes #83472

Scheduler policy configs can no longer be declared multiple times

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Oct 15, 2019
@k8s-ci-robot k8s-ci-robot added sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Oct 15, 2019
@BenTheElder
Copy link
Member

BenTheElder commented Oct 15, 2019 •
edited

/retest
kubernetes-sigs/kind#951 for pull-kubernetes-e2e-kind issue (worked around -- docker hub is down)

@alculquicondor
Copy link
Member

Hi @damemi
From the issue, it seems that we need a validator for the old path, rather than the framework path.
https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/api/validation/validation.go#L34

Also, always remember to add a test.

@damemi
Copy link
Contributor Author

damemi commented Oct 22, 2019

Hi @damemi
From the issue, it seems that we need a validator for the old path, rather than the framework path.
https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/api/validation/validation.go#L34

@alculquicondor Agree that I did take the wrong direction with this PR, but I spoke with @ahg-g and it seems like we may want to do this check when actually registering the custom policies, as in here:

kubernetes/pkg/scheduler/algorithm_factory.go

Line 368 in 6b2b5f2

if policy.Argument.ServiceAntiAffinity != nil {

What do you think? Can probably accomplish the same thing in either spot

@alculquicondor
Copy link
Member

that's right, probably both should do the work. However, the validation.go file has more user-facing semantics.
WDYT @ahg-g?

@ahg-g
Copy link
Member

ahg-g commented Oct 22, 2019

that's right, probably both should do the work. However, the validation.go file has more user-facing semantics.
WDYT @ahg-g?

Aldo's suggestion SGTM.

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 23, 2019
@damemi damemi changed the title Add Scheduler validation check for redeclared plugin configs Add Scheduler validation check for redeclared policy configs Oct 23, 2019
@damemi
Copy link
Contributor Author

damemi commented Oct 23, 2019

Ok, updated this PR to do validation based on the arguments given in the policies in validation.go. Because we have a sort of polymorphism with the argument types, I couldn't think of a much cleaner way to check them, but please review and if you have any ideas for how to refactor this it would be helpful

@ahg-g
Copy link
Member

ahg-g commented Oct 23, 2019

/assign @liu-cong

liu-cong
liu-cong reviewed Oct 23, 2019
View reviewed changes
Copy link
Contributor

@liu-cong liu-cong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a unit test for this

pkg/scheduler/api/validation/validation.go Outdated
// by examining the specified predicate arguments
func validatePredicateRedeclared(predicates map[string]schedulerapi.PredicatePolicy, predicate schedulerapi.PredicatePolicy) error {
var predicateType string
if predicate.Argument.LabelsPresence != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to check whether predicate.Argument is nil or not.

Currently nil argument is valid if the name of the predicate matches a predefined one (I don't know if this is desired or not), see

kubernetes/pkg/scheduler/algorithm_factory.go

Line 281 in e1f86e3

} else if policy.Argument.LabelsPresence != nil {

alculquicondor
alculquicondor reviewed Oct 24, 2019
View reviewed changes
Copy link
Member

@alculquicondor alculquicondor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unit tests, please :)

pkg/scheduler/api/validation/validation.go Outdated Show resolved Hide resolved
ahg-g
ahg-g reviewed Oct 24, 2019
View reviewed changes
Copy link
Member

@ahg-g ahg-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Mike, can you please add a compatibility test as well.

@damemi damemi force-pushed the duplicate-policies branch 2 times, most recently from 4a3de97 to f587f58 Compare October 28, 2019 19:20
@ahg-g
Copy link
Member

ahg-g commented Oct 28, 2019

/retest

1 similar comment
@ahg-g
Copy link
Member

ahg-g commented Oct 29, 2019

/retest

alculquicondor
alculquicondor reviewed Oct 29, 2019
View reviewed changes
pkg/scheduler/api/validation/validation_test.go
expected: errors.New("Predicate 'customPredicate2' redeclares custom predicate 'ServiceAffinity', from:'customPredicate1'"),
},
{
name: "invalid redeclared custom priority",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: a different name from the previous one

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

invalid redeclared custom predicate and invalid redeclared custom priority? They are testing the same thing, but for predicate/priority

@ahg-g
Copy link
Member

ahg-g commented Oct 29, 2019

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 29, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, damemi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 29, 2019
@damemi
Copy link
Contributor Author

damemi commented Oct 29, 2019

/retest

1 similar comment
@ahg-g
Copy link
Member

ahg-g commented Oct 29, 2019

/retest

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit 8d8a068 into kubernetes:master Oct 30, 2019
@k8s-ci-robot k8s-ci-robot added this to the v1.17 milestone Oct 30, 2019
@damemi damemi deleted the duplicate-policies branch October 30, 2019 13:53
@liu-cong liu-cong mentioned this pull request Nov 1, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alculquicondor alculquicondor alculquicondor left review comments

@liu-cong liu-cong liu-cong left review comments

@ahg-g ahg-g ahg-g left review comments

Assignees

@liu-cong liu-cong

@ahg-g ahg-g

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
v1.17
Development

Successfully merging this pull request may close these issues.

Duplicate Custom Predicates/Priorities
7 participants
@damemi @BenTheElder @alculquicondor @ahg-g @k8s-ci-robot @fejta-bot @liu-cong