Skip to content

Commit

Permalink
[SPARK-31073][DOC][FOLLOWUP] Add description for Shuffle Write Time m…
Browse files Browse the repository at this point in the history
…etric in StagePage to web-ui.md

### What changes were proposed in this pull request?

This PR adds description for `Shuffle Write Time` to `web-ui.md`.

### Why are the changes needed?

apache#27837 added `Shuffle Write Time` metric to task metrics summary but it's not documented yet.

### Does this PR introduce any user-facing change?

Yes.
We can see the description for `Shuffle Write Time` in the new `web-ui.html`.
<img width="956" alt="shuffle-write-time-description" src="https://user-images.githubusercontent.com/4736016/78175342-a9722280-7495-11ea-9cc6-62c6f3619aa3.png">

### How was this patch tested?

Built docs by `SKIP_API=1 jekyll build` in `doc` directory and then confirmed `web-ui.html`.

Closes apache#28093 from sarutak/SPARK-31073-doc.

Authored-by: Kousuke Saruta <sarutak@oss.nttdata.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
sarutak authored and Seongjin Cho committed Apr 14, 2020
1 parent a36b243 commit d60fc70
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions docs/web-ui.md
Expand Up @@ -143,6 +143,7 @@ Summary metrics for all task are represented in a table and in a timeline.
* **Shuffle Read Size / Records**. Total shuffle bytes read, includes both data read locally and data read from remote executors.
* **Shuffle Read Blocked Time** is the time that tasks spent blocked waiting for shuffle data to be read from remote machines.
* **Shuffle Remote Reads** is the total shuffle bytes read from remote executors.
* **Shuffle Write Time** is the time that tasks spent writing shuffle data.
* **Shuffle spill (memory)** is the size of the deserialized form of the shuffled data in memory.
* **Shuffle spill (disk)** is the size of the serialized form of the data on disk.

Expand Down

0 comments on commit d60fc70

Please sign in to comment.