feat(docs): KEP-2437-Support Volcano Scheduler in Kubeflow Trainer V2 by Doris-xm · Pull Request #2672 · kubeflow/trainer

Doris-xm · 2025-06-16T13:43:56Z

What this PR does / why we need it:

This PR convert GSoC proposal to KEP. The detailed description of the project is Project 10: Support Volcano Scheduler in Kubeflow Trainer.

Which issue(s) this PR fixes :
Part of #2671

Checklist:

Docs included if any changes are user facing

Signed-off-by: Xinmin Du <2812493086@qq.com>

coveralls · 2025-06-16T13:50:55Z

Pull Request Test Coverage Report for Build 16669262324

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-29.2%) to 0.0%

Totals
Change from base Build 15579727901:	-29.2%
Covered Lines:	0
Relevant Lines:	0

💛 - Coveralls

Signed-off-by: Xinmin Du <2812493086@qq.com>

Electronic-Waste

@Doris-xm Thanks for this great work! I've left my initial comments for you.

/cc @kubeflow/wg-training-leads @astefanutti @rudeigerc

google-oss-prow · 2025-06-23T09:44:18Z

@Electronic-Waste: GitHub didn't allow me to request PR reviews from the following users: rudeigerc.

Note that only kubeflow members and repo collaborators can review this PR, and authors cannot review their own PRs.

Details

In response to this:

@Doris-xm Thanks for this great work! I've left my initial comments for you.

/cc @kubeflow/wg-training-leads @astefanutti @rudeigerc

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Electronic-Waste · 2025-06-23T09:48:28Z

@Doris-xm I also recommend that you could change the title of this PR to either:

KEP-2437: Support Volcano Scheduler in Kubeflow Trainer V2
[GSoC] Project 10: Support Volcano Scheduler in Kubeflow Trainer V2

It will be more clear and neat, I think:)

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

Signed-off-by: Xinmin Du <2812493086@qq.com>

Doris-xm · 2025-06-24T11:20:51Z

/rerun-all

Electronic-Waste

@Doris-xm Thanks for the updates. It looks good to me!

/cc @kubeflow/wg-training-leads @astefanutti @rudeigerc

google-oss-prow · 2025-06-24T11:57:23Z

@Electronic-Waste: GitHub didn't allow me to request PR reviews from the following users: rudeigerc.

Note that only kubeflow members and repo collaborators can review this PR, and authors cannot review their own PRs.

Details

In response to this:

@Doris-xm Thanks for the updates. It looks good to me!

/cc @kubeflow/wg-training-leads @astefanutti @rudeigerc

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: Xinmin Du <2812493086@qq.com>

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

Signed-off-by: Xinmin Du <2812493086@qq.com>

Doris-xm · 2025-07-29T15:26:40Z

Thanks for your time. I've resolved all suggestions above and updated the KEP. Any other suggestions for this PR?
@andreyvelich @Electronic-Waste @rudeigerc

andreyvelich

Thanks for the update @Doris-xm!
Overall lgtm, just a few comments

cc @vsoch in case you want to check KEP to support Gang-Scheduling w/ Volcano for KF Trainer.

Signed-off-by: Xinmin Du <2812493086@qq.com>

Doris-xm · 2025-07-31T07:17:23Z

Based on the latest discussion:

We keep the networkTopology field in podGroupPolicy (volcano): feat(docs): KEP-2437-Support Volcano Scheduler in Kubeflow Trainer V2 #2672 (comment)
We follows the declarative style of Kubernetes APIs. The presence of fields under volcano implies that the Volcano plugin is enabled: feat(docs): KEP-2437-Support Volcano Scheduler in Kubeflow Trainer V2 #2672 (comment)

Any other suggestions for this PR? @andreyvelich @Electronic-Waste @rudeigerc Thanks for your time!

andreyvelich

Thanks a lot for this effort @Doris-xm!
Feel free to un-hold!
/lgtm
/approve

/hold

Signed-off-by: Xinmin Du <2812493086@qq.com>

tenzen-y

Thank you!
/lgtm
/approve

/hold cancel

google-oss-prow · 2025-08-01T10:13:45Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreyvelich, Electronic-Waste, tenzen-y

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [Electronic-Waste,andreyvelich,tenzen-y]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…kubeflow#2672) * doc: KEP creation Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: pre-commit fix Signed-off-by: Xinmin Du <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * fix: syntax err Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: change the example Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: change the explanation for fig. Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: remove unused part Signed-off-by: Xinmin Du <2812493086@qq.com> * feat: add test plan Signed-off-by: Xinmin Du <2812493086@qq.com> * fix Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: update examples Signed-off-by: Xinmin Du <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * fix: reword "integrate with Volcano" to "apply PodGroup" Signed-off-by: Xinmin Du <2812493086@qq.com> * docs: add code block with VolcanoPodPolicySource struct Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: declare permissions with kubebuilder Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: pre-commit fix Signed-off-by: Xinmin Du <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Yuchen Cheng <rudeigerc@gmail.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * fix: inaccurate description Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: modify code example & remove unnecessary installation step Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: update test plan Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: a few nits Signed-off-by: Xinmin Du <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * fix: code block format Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: omit minTaskMember Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: configure queue in annotation Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: trim trail whitespace Signed-off-by: Xinmin Du <2812493086@qq.com> * Update docs/proposals/2437-volcano-scheduler/README.md Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> * fix: Sedna case in Motivation Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: configure priorityClassName from pod spec Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: rm PodGroupInterface Signed-off-by: Xinmin Du <2812493086@qq.com> * fix: explain `queue` override Signed-off-by: Xinmin Du <2812493086@qq.com> * feat: Volcano Scheduler Plugin Configuration Signed-off-by: Xinmin Du <2812493086@qq.com> --------- Signed-off-by: Xinmin Du <2812493086@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com> Co-authored-by: Shao Wang <2690692950@qq.com> Co-authored-by: Yuchen Cheng <rudeigerc@gmail.com> Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

doc: KEP creation

7a3afdd

Signed-off-by: Xinmin Du <2812493086@qq.com>

google-oss-prow Bot requested review from jinchihe and kuizhiqing June 16, 2025 13:44

google-oss-prow Bot added the size/L label Jun 16, 2025

Doris-xm mentioned this pull request Jun 16, 2025

[GSoC] Project 10: Support Volcano Scheduler in Kubeflow Trainer #2671

Closed

2 tasks

fix: pre-commit fix

bbc1cee

Signed-off-by: Xinmin Du <2812493086@qq.com>

Doris-xm force-pushed the KEP-volcano-scheduler branch from 2c6f9c6 to bbc1cee Compare June 16, 2025 14:04

Electronic-Waste reviewed Jun 23, 2025

View reviewed changes

google-oss-prow Bot requested review from a team and astefanutti June 23, 2025 09:44

Doris-xm and others added 11 commits June 24, 2025 16:20

Update docs/proposals/2437-volcano-scheduler/README.md

f423403

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

Update docs/proposals/2437-volcano-scheduler/README.md

1240db2

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

Update docs/proposals/2437-volcano-scheduler/README.md

be9dcf5

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

Update docs/proposals/2437-volcano-scheduler/README.md

484dcb0

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

Update docs/proposals/2437-volcano-scheduler/README.md

55ef7e8

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

Update docs/proposals/2437-volcano-scheduler/README.md

2a73c87

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

Update docs/proposals/2437-volcano-scheduler/README.md

ab203ae

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

Update docs/proposals/2437-volcano-scheduler/README.md

cfd5a9a

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

fix: syntax err

b6428ba

Signed-off-by: Xinmin Du <2812493086@qq.com>

fix: change the example

fbe8f5f

Signed-off-by: Xinmin Du <2812493086@qq.com>

fix: change the explanation for fig.

cd0aee3

Signed-off-by: Xinmin Du <2812493086@qq.com>

Doris-xm changed the title ~~KEP-2437: Creating Kubeflow Enhancement Proposal~~ KEP-2437: Support Volcano Scheduler in Kubeflow Trainer V2 Jun 24, 2025

Doris-xm added 2 commits June 24, 2025 18:47

fix: remove unused part

f5c8b3e

Signed-off-by: Xinmin Du <2812493086@qq.com>

feat: add test plan

52641fa

Signed-off-by: Xinmin Du <2812493086@qq.com>

Electronic-Waste reviewed Jun 24, 2025

View reviewed changes

Doris-xm changed the title ~~KEP-2437: Support Volcano Scheduler in Kubeflow Trainer V2~~ feat(docs): KEP-2437-Support Volcano Scheduler in Kubeflow Trainer V2 Jul 25, 2025

Doris-xm and others added 4 commits July 29, 2025 23:02

fix: configure queue in annotation

61d74df

Signed-off-by: Xinmin Du <2812493086@qq.com>

fix: trim trail whitespace

e785c91

Signed-off-by: Xinmin Du <2812493086@qq.com>

Update docs/proposals/2437-volcano-scheduler/README.md

eaa8d34

Co-authored-by: Shao Wang <2690692950@qq.com> Signed-off-by: Du Xinmin <2812493086@qq.com>

fix: Sedna case in Motivation

9c49ec8

Signed-off-by: Xinmin Du <2812493086@qq.com>

andreyvelich reviewed Jul 30, 2025

View reviewed changes

Comment thread docs/proposals/2437-volcano-scheduler/README.md Outdated

Comment thread docs/proposals/2437-volcano-scheduler/README.md Outdated

Comment thread docs/proposals/2437-volcano-scheduler/README.md

fix: configure priorityClassName from pod spec

b578bc3

Signed-off-by: Xinmin Du <2812493086@qq.com>

andreyvelich reviewed Jul 31, 2025

View reviewed changes

google-oss-prow Bot assigned andreyvelich Jul 31, 2025

google-oss-prow Bot added the lgtm label Jul 31, 2025

tenzen-y reviewed Jul 31, 2025

View reviewed changes

Comment thread docs/proposals/2437-volcano-scheduler/README.md Outdated

tenzen-y reviewed Jul 31, 2025

View reviewed changes

Comment thread docs/proposals/2437-volcano-scheduler/README.md

tenzen-y reviewed Jul 31, 2025

View reviewed changes

Comment thread docs/proposals/2437-volcano-scheduler/README.md

tenzen-y reviewed Jul 31, 2025

View reviewed changes

Comment thread docs/proposals/2437-volcano-scheduler/README.md

Doris-xm added 3 commits August 1, 2025 11:04

fix: rm PodGroupInterface

9e9af9a

Signed-off-by: Xinmin Du <2812493086@qq.com>

fix: explain queue override

c33e18d

Signed-off-by: Xinmin Du <2812493086@qq.com>

feat: Volcano Scheduler Plugin Configuration

49464cc

Signed-off-by: Xinmin Du <2812493086@qq.com>

google-oss-prow Bot removed the lgtm label Aug 1, 2025

tenzen-y reviewed Aug 1, 2025

View reviewed changes

Comment thread docs/proposals/2437-volcano-scheduler/README.md

google-oss-prow Bot removed the do-not-merge/hold label Aug 1, 2025

google-oss-prow Bot assigned tenzen-y Aug 1, 2025

google-oss-prow Bot added the lgtm label Aug 1, 2025

google-oss-prow Bot merged commit cfc25d5 into kubeflow:master Aug 1, 2025
18 checks passed

google-oss-prow Bot added this to the v2.1 milestone Aug 1, 2025

Electronic-Waste mentioned this pull request Aug 1, 2025

feat: KEP-2437 - PodGroup Creation for Volcano Scheduler #2729

Merged

1 task

Doris-xm mentioned this pull request Oct 18, 2025

fix(docs): correct example usage in KEP-2437-Support-Volcano-Scheduler #2898

Merged

1 task

Uh oh!

Conversation

Doris-xm commented Jun 16, 2025

Uh oh!

coveralls commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 16669262324

Warning: This coverage report may be inaccurate.

Details

💛 - Coveralls

Uh oh!

Electronic-Waste left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

google-oss-prow Bot commented Jun 23, 2025

Uh oh!

Electronic-Waste commented Jun 23, 2025

Uh oh!

Doris-xm commented Jun 24, 2025

Uh oh!

Electronic-Waste left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

google-oss-prow Bot commented Jun 24, 2025

Uh oh!

Doris-xm commented Jul 29, 2025

Uh oh!

andreyvelich left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Doris-xm commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andreyvelich left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tenzen-y left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

google-oss-prow Bot commented Aug 1, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

coveralls commented Jun 16, 2025 •

edited

Loading

Doris-xm commented Jul 31, 2025 •

edited

Loading