[SPARK-5768] [Web UI] Fix for incorrect memory in Spark UI #6972
Conversation
Pulling functionality from apache spark
pull latest from apache spark
Pulling functionality from apache spark
| @@ -35,6 +35,9 @@ private[spark] object ToolTips { | |||
|
|
|||
| val OUTPUT = "Bytes and records written to Hadoop." | |||
|
|
|||
| val STORAGE_MEMORY = | |||
| "Total memory used for caching of the total available memory for caching.This is the storage memory used on the executor." | |||
There was a problem hiding this comment.
I think, storage memory is used by not only cached-RDD right?
There was a problem hiding this comment.
I suggest something like "Memory used / total available for storage of data like RDD partitions cached in memory"?
|
ok to test |
|
Test build #35710 has finished for PR 6972 at commit
|
|
@rekhajoshm , This test failure is due to style. L39 exceeds 100 characters. Could you fix it? |
|
Thanks @andrewor14 @sarutak @srowen for quick inputs and review comments. |
| val STORAGE_MEMORY = | ||
| """Memory used / total available memory for storage of data like RDD partitions cached in | ||
| memory. """ | ||
|
|
There was a problem hiding this comment.
Why is this text is enclosed by triple-double-quote? It bring needless new line.
How about make the text like as follows?
"Memory used / total available memory for storage of data " +
"like RDD partitions cached in memory. "
|
I left one minor comment, otherwise it LGTM. |
|
thanks @sarutak done as suggested. |
|
Test build #35750 has finished for PR 6972 at commit
|
|
Test build #35753 has finished for PR 6972 at commit
|
|
LGTM |
|
Merging |
Fix for incorrect memory in Spark UI as per SPARK-5768 Author: Joshi <rekhajoshm@gmail.com> Author: Rekha Joshi <rekhajoshm@gmail.com> Closes #6972 from rekhajoshm/SPARK-5768 and squashes the following commits: b678a91 [Joshi] Fix for incorrect memory in Spark UI 2fe53d9 [Joshi] Fix for incorrect memory in Spark UI eb823b8 [Joshi] SPARK-5768: Fix for incorrect memory in Spark UI 0be142d [Rekha Joshi] Merge pull request #3 from apache/master 106fd8e [Rekha Joshi] Merge pull request #2 from apache/master e3677c9 [Rekha Joshi] Merge pull request #1 from apache/master (cherry picked from commit 085a721) Signed-off-by: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
Fix for incorrect memory in Spark UI as per SPARK-5768