Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

volumebinding: scheduler queueing hints - PersistentVolumeClaim #124959

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

volumebinding: scheduler queueing hints - PersistentVolumeClaim #124959

wants to merge 7 commits into from
+156 −1

Conversation

bells17
Copy link
Contributor

@bells17 bells17 commented May 20, 2024 •
edited

What type of PR is this?

/kind feature

What this PR does / why we need it:

kube-scheduler implements scheduling hints for the VolumeBinding plugin.
The scheduling hints allow the scheduler to determine whether to retry or skip scheduling a Pod based on the changes made to the PersistentVolumeClaim resource referenced by the plugin.

Which issue(s) this PR fixes:

Part of #118893
KEP: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/4247-queueinghint/README.md
Base PR: #124939

Special notes for your reviewer:

Fields Impacting QueueingHintFn

PersistentVolume (PV) is not included in this table because it can undergo extensive changes when a conversion is performed by csi-translation-lib.

resource field Referenced in PreFilter+Filter? Admission Overwrite Prevention Config Need to Check Changes in QHint?
PersistentVolumeClaim(PVC) .metadata.labels x x x
PersistentVolumeClaim(PVC) .metadata.annotations o o o
PersistentVolumeClaim(PVC) .spec.accessModes o x o
PersistentVolumeClaim(PVC) .spec.selector o o x
PersistentVolumeClaim(PVC) .spec.resources o o x
PersistentVolumeClaim(PVC) .spec.volumeName o o x
PersistentVolumeClaim(PVC) .spec.storageClassName o x o
PersistentVolumeClaim(PVC) .spec.volumeMode x o x
PersistentVolumeClaim(PVC) .spec.dataSource x o x
PersistentVolumeClaim(PVC) .spec.dataSourceRef x o x
PersistentVolumeClaim(PVC) .spec.volumeAttributesClassName x o x
PersistentVolumeClaim(PVC) .status.phase o x o
PersistentVolumeClaim(PVC) .status.accessModes x x x
PersistentVolumeClaim(PVC) .status.capacity x x x
PersistentVolumeClaim(PVC) .status.conditions x x x
PersistentVolumeClaim(PVC) .status.allocatedResources x x x
PersistentVolumeClaim(PVC) .status.allocatedResourceStatuses x x x
PersistentVolumeClaim(PVC) .status.currentVolumeAttributesClassName x x x
PersistentVolumeClaim(PVC) .status.modifyVolumeStatus x x x

ref(ja): https://zenn.dev/bells17/scraps/65bd6891012bdc

Does this PR introduce a user-facing change?

kube-scheduler implements scheduling hints for the VolumeBinding plugin.
The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the VolumeBinding plugin only if a new resource referenced by the plugin was created or an existing resource referenced by the plugin was updated.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels May 20, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: bells17
Once this PR has been reviewed and has the lgtm label, please assign msau42 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. sig/storage Categorizes an issue or PR as relevant to SIG Storage. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels May 20, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels May 20, 2024
@bells17
Copy link
Contributor Author

bells17 commented May 20, 2024

/cc @sanposhiho @utam0k

@bells17 bells17 mentioned this pull request May 20, 2024
Open
12 tasks
@sanposhiho
Copy link
Member

/kind feature

@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. and removed do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels May 20, 2024
@bells17
Copy link
Contributor Author

bells17 commented May 20, 2024

/retest

@bells17 bells17 force-pushed the qhint-volume-binding-pvc branch 2 times, most recently from bdae41a to c98c26a Compare May 21, 2024 04:37
@bells17 bells17 mentioned this pull request May 23, 2024
Closed
@bells17
Copy link
Contributor Author

bells17 commented May 25, 2024

/retest

@bells17
Copy link
Contributor Author

bells17 commented May 26, 2024

/retest

@bells17 bells17 requested a review from sanposhiho May 26, 2024 13:48
@bells17
Copy link
Contributor Author

bells17 commented May 26, 2024

/retest

@bells17
Copy link
Contributor Author

bells17 commented May 26, 2024

@sanposhiho Thank you for your review. I have made the necessary changes, so please take another look.

@carlory
Copy link
Member

carlory commented May 27, 2024

/lgtm
(from storage view)

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 27, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 73a7afd4eae8c58c5eef61e7c30d92b2126b9d51

sanposhiho
sanposhiho reviewed May 28, 2024
View reviewed changes
Copy link
Member

@sanposhiho sanposhiho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only nits

pkg/scheduler/framework/plugins/volumebinding/volume_binding.go Outdated
)

if pod.Namespace != newPVC.Namespace {
logger.V(5).Info("PersistentVolumeClaim was created or updated, but it doesn't make this pod schedulable")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logger.V(5).Info("PersistentVolumeClaim was created or updated, but it doesn't make this pod schedulable")
logger.V(5).Info("PersistentVolumeClaim was created or updated, but it doesn't make this pod schedulable because the PVC belongs to a different namespace")

pkg/scheduler/framework/plugins/volumebinding/volume_binding.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 29, 2024
@k8s-ci-robot
Copy link
Contributor

New changes are detected. LGTM label has been removed.

@bells17
Copy link
Contributor Author

bells17 commented May 29, 2024

/retest

@bells17 bells17 requested a review from sanposhiho May 30, 2024 02:21
@bells17 bells17 changed the title volumebinding: scheduler queueing hints - PersistentVolumeClaim volumebinding: scheduler queueing hints May 31, 2024
sanposhiho
sanposhiho reviewed May 31, 2024
View reviewed changes
pkg/scheduler/framework/plugins/volumebinding/volume_binding.go
// We bind PVCs with PVs, so any changes may make the pods schedulable.
{Event: framework.ClusterEvent{Resource: framework.PersistentVolumeClaim, ActionType: framework.Add | framework.Update}},
// QHintFn is added only for PersistentVolumeClaim because:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it really "only" pvc? I haven't looked at @YamasouA's PRs deeply yet though, they look alrighty; no reference to other resources.

@bells17 bells17 changed the title volumebinding: scheduler queueing hints volumebinding: scheduler queueing hints - PersistentVolumeClaim May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sanposhiho sanposhiho sanposhiho left review comments

@humblec humblec Awaiting requested review from humblec

@mauriciopoppe mauriciopoppe Awaiting requested review from mauriciopoppe

@utam0k utam0k Awaiting requested review from utam0k

@carlory carlory Awaiting requested review from carlory

@jsafrane jsafrane Awaiting requested review from jsafrane

@xing-yang xing-yang Awaiting requested review from xing-yang

Assignees

@carlory carlory

Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. sig/storage Categorizes an issue or PR as relevant to SIG Storage. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@bells17 @k8s-ci-robot @sanposhiho @carlory @utam0k