Topology Aware Volume Scheduling

[msau42]

Feature Description

• One-line feature description (can be used as a release note): Make the scheduler aware of a Pod's volume's topology constraints, such as zone or node.
• Primary contact (assignee): @msau42
• Responsible SIGs: @kubernetes/sig-storage-feature-requests @kubernetes/sig-scheduling-feature-requests
• Design proposal link (community repo):Volume topology aware scheduling design community#1054, Add more details to volume scheduling design community#1168
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
  • @jsafrane
  • @bsalamat
• Approver (likely from SIG/area to which feature belongs):
  • @thockin
• Feature target (which target equals to which milestone):
  • Alpha release target (x.y): 1.9
  • Beta release target (x.y): 1.10
  • Stable release target (x.y): 1.13

/assign

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[msau42]

/remove-lifecycle-stale
/lifecycle frozen

I'm hoping to target beta in 1.10

[idvoretskyi]

@msau42 1.10 - beta, right?

[msau42]

Yes, at least I will try to.

[msau42]

1.10 Beta PR has been merged: kubernetes/kubernetes#59391

1.11 design for dynamic provisioning is here: kubernetes/community#1857

[ku-s-h]

@msau42 In my use case, the statefulset does not automatically schedule pods on the same node as the pv(local storage). To overcome this I have written a custom scheduler that schedules the pod on the node which has it's pv. This also works in the case where the pod goes down and needs to be brought up on the exact node that has it's pv.
Will the changes planned for 1.11 eliminate the need for a custom scheduler or is there a better way to do this?

[msau42]

@mightwork the current feature should already be scheduling your pod to the correct node. Can you open an issue with the details of your PV and ping me?

[justaugustus]

@msau42
Any plans for this in 1.11?

If so, can you please ensure the feature is up-to-date with the appropriate:

• Description
• Milestone
• Assignee(s)
• Labels:
  • stage/{alpha,beta,stable}
  • sig/*
  • kind/feature

cc @idvoretskyi

[msau42]

Plan is to remain beta in 1.11.

#561 is tracking the dynamic provisioning work

[msau42]

Plan for 1.13 is to go GA

[kacole2]

/milestone v1.13

[AishSundar]

@msau what work if left in k/k for this feature to go to GA in 1.13? Is there a list of pending PRs (code, tests and docs) that we can track on our end?

/stage stable

[AishSundar]

@kacole2 I updated the stage to be stable as per Michelle's comment here

[msau42]

Remaining tasks:

• Add more e2e test cases: Add more e2es for topology aware dynamic provisioning kubernetes#68517
• Add metrics: Add metrics for volume scheduling operations kubernetes#59529
• Feature gate change to GA: Move volume scheduling feature to GA kubernetes#70673
• Docs: Doc updates for volume scheduling GA website#10736

[npentrel]

Hi @msau42, I'm Naomi and working on docs for the 1.13 release. Could you open up a docs PR against the dev-1.13 branch as a placeholder for the needed docs updates?

[msau42]

Docs pr here: kubernetes/website#10736

[guineveresaenger]

Hi @msau42 - I'm Enhancements Shadow for 1.13. Could you please update the release team with the likelihood of this enhancement making the 1.13 release? When we don't see any updates in a few days, we would appreciate an update.

Code slush begins on 11/9 and code freeze is 11/15.

Thank you!

[msau42]

Yes, this is on track to meet code freeze.

e2e PR is close to being merged:
kubernetes/kubernetes#70362

I will open up the feature gate PR this week.

[AishSundar]

@msau42 is the e2e test PR the only pending PR for this enhancement? I added milestone and priority to the PR so it can merge in Slush.

Is there a feature gate PR open?

[msau42]

I updated the checklist above. Just docs are remaining

[claurence]

@msau42 Hi I'm the enhancements lead for 1.14 - It looks like this enhancement graduated to stable - is this issue still open? Is there work needed for the 1.14 release?

[msau42]

I think we are done with feature tracking. There are some cleanup items that I have already opened issues for.
/close

[k8s-ci-robot]

@msau42: Closing this issue.

In response to this:

I think we are done with feature tracking. There are some cleanup items that I have already opened issues for.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[abdennour]

Actually, this is also a pain with statefulsets running on top of Amazon EKS Cluster.
Hopefully, the fix comes from kubernetes