Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-33891][DOCS][CORE] Update dynamic allocation related documents #30906

Closed
wants to merge 2 commits into from
Closed

[SPARK-33891][DOCS][CORE] Update dynamic allocation related documents #30906

wants to merge 2 commits into from

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Dec 23, 2020

What changes were proposed in this pull request?

This PR aims to update the followings.

  • Remove the outdated requirement for spark.shuffle.service.enabled in configuration.md
  • Dynamic allocation section in job-scheduling.md

Why are the changes needed?

To make the document up-to-date.

Does this PR introduce any user-facing change?

No, it's a documentation update.

How was this patch tested?

Manual.

BEFORE
Screen Shot 2020-12-23 at 2 22 04 AM
Screen Shot 2020-12-23 at 2 22 34 AM

AFTER
Screen Shot 2020-12-23 at 2 25 36 AM
Screen Shot 2020-12-23 at 2 21 13 AM

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-33891][DOCS] Update dynamic allocation related documents [SPARK-33891][DOCS][K8S] Update dynamic allocation related documents Dec 23, 2020
@dongjoon-hyun
Copy link
Member Author

dongjoon-hyun commented Dec 23, 2020

Could you review this please, @HyukjinKwon ?

@SparkQA
Copy link

SparkQA commented Dec 23, 2020

Test build #133296 has finished for PR 30906 at commit fc85d03.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@github-actions github-actions bot added the DOCS label Dec 23, 2020
Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@HyukjinKwon
Copy link
Member

BTW, @dongjoon-hyun, Shall we fix the tag in PR title? I think it's not specific to Kubernates.

@HyukjinKwon HyukjinKwon changed the title [SPARK-33891][DOCS][K8S] Update dynamic allocation related documents [SPARK-33891][DOCS][CORE] Update dynamic allocation related documents Dec 23, 2020
@HyukjinKwon
Copy link
Member

Merged to master, branch-3.1 and branch-3.0.

HyukjinKwon pushed a commit that referenced this pull request Dec 23, 2020
### What changes were proposed in this pull request?

This PR aims to update the followings.
- Remove the outdated requirement for `spark.shuffle.service.enabled` in `configuration.md`
- Dynamic allocation section in `job-scheduling.md`

### Why are the changes needed?

To make the document up-to-date.

### Does this PR introduce _any_ user-facing change?

No, it's a documentation update.

### How was this patch tested?

Manual.

**BEFORE**
![Screen Shot 2020-12-23 at 2 22 04 AM](https://user-images.githubusercontent.com/9700541/102986441-ae647f80-44c5-11eb-97a3-87c2d368952a.png)
![Screen Shot 2020-12-23 at 2 22 34 AM](https://user-images.githubusercontent.com/9700541/102986473-bcb29b80-44c5-11eb-8eae-6802001c6dfa.png)

**AFTER**
![Screen Shot 2020-12-23 at 2 25 36 AM](https://user-images.githubusercontent.com/9700541/102986767-2df24e80-44c6-11eb-8540-e74856a4c313.png)
![Screen Shot 2020-12-23 at 2 21 13 AM](https://user-images.githubusercontent.com/9700541/102986366-8e34c080-44c5-11eb-8054-1efd07c9458c.png)

Closes #30906 from dongjoon-hyun/SPARK-33891.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
(cherry picked from commit 47d1aa4)
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
HyukjinKwon pushed a commit that referenced this pull request Dec 23, 2020
### What changes were proposed in this pull request?

This PR aims to update the followings.
- Remove the outdated requirement for `spark.shuffle.service.enabled` in `configuration.md`
- Dynamic allocation section in `job-scheduling.md`

### Why are the changes needed?

To make the document up-to-date.

### Does this PR introduce _any_ user-facing change?

No, it's a documentation update.

### How was this patch tested?

Manual.

**BEFORE**
![Screen Shot 2020-12-23 at 2 22 04 AM](https://user-images.githubusercontent.com/9700541/102986441-ae647f80-44c5-11eb-97a3-87c2d368952a.png)
![Screen Shot 2020-12-23 at 2 22 34 AM](https://user-images.githubusercontent.com/9700541/102986473-bcb29b80-44c5-11eb-8eae-6802001c6dfa.png)

**AFTER**
![Screen Shot 2020-12-23 at 2 25 36 AM](https://user-images.githubusercontent.com/9700541/102986767-2df24e80-44c6-11eb-8540-e74856a4c313.png)
![Screen Shot 2020-12-23 at 2 21 13 AM](https://user-images.githubusercontent.com/9700541/102986366-8e34c080-44c5-11eb-8054-1efd07c9458c.png)

Closes #30906 from dongjoon-hyun/SPARK-33891.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
(cherry picked from commit 47d1aa4)
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
@dongjoon-hyun
Copy link
Member Author

Thank you, @HyukjinKwon !

@dongjoon-hyun dongjoon-hyun deleted the SPARK-33891 branch December 23, 2020 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants