[SPARK-12716] [Web UI] Add a TOTALS row to the Executors Web UI #10668
Conversation
|
Screenshots: |
|
Test build #49035 has finished for PR 10668 at commit
|
|
@srowen @tgravescs This is for the second half of SPARK-1832 that you reported if you want to take a look |
|
thanks for working on this. After seeing it I'm wondering if it would look better as a separate table just for the summary. Similar to the "Summary Metrics for X Completed Tasks" for a stage. what do you guys think? copy @JoshRosen |
|
@tgravescs I can put together an implementation of that to compare to, I don't know which would look better though, The stages summary doesn't have the same fields as the tables below it like this would, might look odd. |
|
Here's a screenshot of @tgravescs summary table idea. [Note: this code is NOT committed] What do people think of this alternative? Opinions on how to improve it? I think we need to add a subhead for the original table, but I'm unsure of what it should say |
|
personally I like this better because its always there and not moving when sorting. One other reason I was going for a separate table is I filed another jira to show killed executors. I was thinking it might work as a nice summary for that where you could show active executors, inactive, as well as summaries across. |
|
I just looked at that other PR and this will look great once that get's merged, I'll update the code to match the above screenshot |
This reverts commit f0a725d.
|
Test build #49245 has finished for PR 10668 at commit
|
|
Do we want to wait for PR #10058 and then add an ACTIVE and DEAD row to the the table? Or Is this good to go? #10058 seems stale, but I feel like the table looks off with just one row. Also I think adding a header to the original table would look better but I personally can't think of what the header would say other than "Executors" which is already the page header, opinions? |
|
No we don't need to wait for that. I actually filed a new jira for what looks like the same thing. You have "Totals for 5 Executors" on the top table so how about the bottom one we just put "Active Executors" since it doesn't show ones that are killed anyway. |
| <div class="span12"> | ||
| <ul class="unstyled"> | ||
| <li><strong>Memory:</strong> | ||
| {Utils.bytesToString(memUsed)} Used |
There was a problem hiding this comment.
memUsed and maxMem locals in this function aren't being used anymore so lets remove them.
|
|
|
Test build #49426 has finished for PR 10668 at commit
|
|
Test build #49423 has finished for PR 10668 at commit
|
|
+1, thanks @ajbozarth |
|
Oops sorry, it looks like it doesn't merge cleanly anymore. can you upmerge again. |
|
The tests finished out of order, check the commit numbers. I thought that first too. |
…otals Table ## What changes were proposed in this pull request? Now that dead executors are shown in the executors table (#10058) the totals table is updated to include the separate totals for alive and dead executors as well as the current total, as originally discussed in #10668 ## How was this patch tested? Manually verified by running the Standalone Web UI in the latest Safari and Firefox ESR Author: Alex Bozarth <ajbozart@us.ibm.com> Closes #11381 from ajbozarth/spark13459.
…otals Table ## What changes were proposed in this pull request? Now that dead executors are shown in the executors table (apache#10058) the totals table is updated to include the separate totals for alive and dead executors as well as the current total, as originally discussed in apache#10668 ## How was this patch tested? Manually verified by running the Standalone Web UI in the latest Safari and Firefox ESR Author: Alex Bozarth <ajbozart@us.ibm.com> Closes apache#11381 from ajbozarth/spark13459.
Added a Totals table to the top of the page to display the totals of each applicable column in the executors table.
Old Description:
Created a TOTALS row containing the totals of each column in the executors UI. By default the TOTALS row appears at the top of the table. When a column is sorted the TOTALS row will always sort to either the top or bottom of the table.