[SPARK-30664][Web UI] Add optional metrics to all-stages page#27375
[SPARK-30664][Web UI] Add optional metrics to all-stages page#27375EnricoMi wants to merge 2 commits intoapache:masterfrom
Conversation
|
Can one of the admins verify this patch? |
EnricoMi
changed the title
| } | ||
| } | ||
| </td> ++ | ||
| {if (metrics.contains("Input")) { |
There was a problem hiding this comment.
It could be interesting to extract all this string to constants to avoid to look for all places if the string changes.
Don't you think?
There was a problem hiding this comment.
good in general but I think out-of-scope for this PR
| } else { | ||
| Seq.empty | ||
| }} ++ | ||
| {if (metrics.contains("Output")) { |
There was a problem hiding this comment.
Could we find a way to avoid all this duplicate code? Something like iterate over optionalHeaders.keys or something like that.
EnricoMi
changed the title
|
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. |
|
Can someone please reopen this PR? |
4 participants
What changes were proposed in this pull request?
Adding more metrics to the all-stages page, some of those available at the individual stages pages, namely
Peak Execution Memory,Spill (Memory),Spill (Disk)andGC Time.Why are the changes needed?
Finding stages that perform poorly is difficult when you have hundreds or thousands of stages. Sorting all stages by those extra metrics helps to identify those stages.
Does this PR introduce any user-facing change?
Yes, in the Web UI, not in the API.
How was this patch tested?
Manual tests.