Skip to content

Conversation

@dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Jan 21, 2023

What changes were proposed in this pull request?

This PR aims to upgrade Volcano from 1.5.1 to 1.7.0.

Why are the changes needed?

Volcano 1.7.0 finally provides multi-arch image and K8s 1.25.

Does this PR introduce any user-facing change?

No. This is a doc-only change.

How was this patch tested?

Manually verify with Volcano 1.7.0.

$ build/sbt -Psparkr -Pkubernetes -Pvolcano -Pkubernetes-integration-tests -Dtest.exclude.tags=minikube,local -Dtest.include.tags=volcano -Dspark.kubernetes.test.deployMode=docker-desktop "kubernetes-integration-tests/test"
...
[info] VolcanoSuite:
[info] - Run SparkPi with volcano scheduler (8 seconds, 331 milliseconds)
[info] - SPARK-38187: Run SparkPi Jobs with minCPU (30 seconds, 563 milliseconds)
[info] - SPARK-38187: Run SparkPi Jobs with minMemory (27 seconds, 505 milliseconds)
[info] - SPARK-38188: Run SparkPi jobs with 2 queues (only 1 enabled) (12 seconds, 141 milliseconds)
[info] - SPARK-38188: Run SparkPi jobs with 2 queues (all enabled) (21 seconds, 247 milliseconds)
[info] - SPARK-38423: Run driver job to validate priority order (21 seconds, 516 milliseconds)
[info] YuniKornSuite:
[info] Run completed in 2 minutes, 37 seconds.
[info] Total number of tests run: 6
[info] Suites: completed 3, aborted 0
[info] Tests: succeeded 6, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 412 s (06:52), completed Jan 21, 2023, 8:04:01 AM

@dongjoon-hyun dongjoon-hyun marked this pull request as ready for review January 21, 2023 16:08
@dongjoon-hyun dongjoon-hyun changed the title [SPARK-42150][K8S][DOCS] Upgrade Volcano to 1.7.0 [SPARK-42150][K8S][DOCS] Upgrade Volcano to 1.7.0 Jan 21, 2023
@dongjoon-hyun
Copy link
Member Author

Since this is a doc-only PR, GitHub action result is irrelevant.

cc @Yikun

@dongjoon-hyun
Copy link
Member Author

Could you review this when you have some time, @viirya ?

@dongjoon-hyun
Copy link
Member Author

Could you review this, @gengliangwang ?

# arm64:
kubectl delete -f https://raw.githubusercontent.com/volcano-sh/volcano/v1.5.1/installer/volcano-development-arm64.yaml

# Cleanup Volcano webhook
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just double confirm: the cleanup webhook is no longer needed?

Copy link
Member Author

@dongjoon-hyun dongjoon-hyun Jan 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, It's included by the original YAML now. So, if we try to clean up here again, it shows error messages because there is no such resource here.

@dongjoon-hyun
Copy link
Member Author

Thank you so much, @gengliangwang . Merged to master.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-42150 branch January 22, 2023 00:49
Copy link
Member

@viirya viirya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@Yikun
Copy link
Member

Yikun commented Jan 22, 2023

@dongjoon-hyun Thanks! Late LGTM.

dongjoon-hyun added a commit that referenced this pull request Feb 22, 2023
### What changes were proposed in this pull request?

This is a follow-up of #39690.

### Why are the changes needed?

To be consistent across multiple docs.

### Does this PR introduce _any_ user-facing change?

No, this is a doc-only change.

### How was this patch tested?

Manual review.

Closes #40131 from dongjoon-hyun/SPARK-42150-2.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
dongjoon-hyun added a commit that referenced this pull request Feb 22, 2023
### What changes were proposed in this pull request?

This is a follow-up of #39690.

### Why are the changes needed?

To be consistent across multiple docs.

### Does this PR introduce _any_ user-facing change?

No, this is a doc-only change.

### How was this patch tested?

Manual review.

Closes #40131 from dongjoon-hyun/SPARK-42150-2.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 70e6989)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
snmvaughan pushed a commit to snmvaughan/spark that referenced this pull request Jun 20, 2023
### What changes were proposed in this pull request?

This is a follow-up of apache#39690.

### Why are the changes needed?

To be consistent across multiple docs.

### Does this PR introduce _any_ user-facing change?

No, this is a doc-only change.

### How was this patch tested?

Manual review.

Closes apache#40131 from dongjoon-hyun/SPARK-42150-2.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 70e6989)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants