Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-2016][WebUI]RDD partition table pagination for the RDD Page #7692

Closed
wants to merge 2 commits into from
Closed

[SPARK-2016][WebUI]RDD partition table pagination for the RDD Page #7692

wants to merge 2 commits into from

Conversation

carsonwang
Copy link
Contributor

Add pagination for the RDD page to avoid unresponsive UI when the number of the RDD partitions is large.
Before:
rddpagebefore
After:
rddpageafter

@SparkQA
Copy link

SparkQA commented Jul 27, 2015

Test build #38531 timed out for PR 7692 at commit 612c18c after a configured wait of 175m.

@carsonwang
Copy link
Contributor Author

retest this please

@SparkQA
Copy link

SparkQA commented Jul 28, 2015

Test build #126 has finished for PR 7692 at commit 612c18c.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@carsonwang
Copy link
Contributor Author

retest this please

@SparkQA
Copy link

SparkQA commented Jul 28, 2015

Test build #131 has finished for PR 7692 at commit 612c18c.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jul 28, 2015

Test build #38671 has finished for PR 7692 at commit 612c18c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor

@zsxwing could you review this?

@andrewor14
Copy link
Contributor

@carsonwang would you mind posting a screenshot of what this looks like before and after?

@carsonwang
Copy link
Contributor Author

Screenshots attached.

@JoshRosen
Copy link
Contributor

Bump. @zsxwing (or maybe @sarutak if you have cycles), let's see about pulling this in for 1.5?

@sarutak
Copy link
Member

sarutak commented Aug 3, 2015

O.K, I'll take a look soon.

sarutak
sarutak reviewed Aug 3, 2015
View reviewed changes
core/src/main/scala/org/apache/spark/ui/storage/RDDPage.scala
import javax.servlet.http.HttpServletRequest

import scala.xml.Node
import scala.xml.{Unparsed, Node}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you sort this line with dictionary order?

@carsonwang
Copy link
Contributor Author

Thanks @sarutak . The style issues are fixed.

@sarutak
Copy link
Member

sarutak commented Aug 4, 2015

Works well in my machine.

@SparkQA
Copy link

SparkQA commented Aug 4, 2015

Test build #39636 has finished for PR 7692 at commit 03c7168.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

zsxwing
zsxwing reviewed Aug 4, 2015
View reviewed changes
core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
@@ -1218,7 +1216,7 @@ private[ui] class TaskPagedTable(
Seq(("Errors", ""))

if (!taskHeadersAndCssClasses.map(_._1).contains(sortColumn)) {
new IllegalArgumentException(s"Unknown column: $sortColumn")
throw new IllegalArgumentException(s"Unknown column: $sortColumn")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Thanks!

@zsxwing
Copy link
Member

zsxwing commented Aug 4, 2015

LGTM

@sarutak
Copy link
Member

sarutak commented Aug 4, 2015

O.K. Merging into master and branch-1.5. Thanks @carsonwang !

asfgit pushed a commit that referenced this pull request Aug 4, 2015
@carsonwang @sarutak
[SPARK-2016] [WEBUI] RDD partition table pagination for the RDD Page
45c8d2b 
Add pagination for the RDD page to avoid unresponsive UI when the number of the RDD partitions is large.
Before:
![rddpagebefore](https://cloud.githubusercontent.com/assets/9278199/8951533/3d9add54-3601-11e5-99d0-5653b473c49b.png)
After:
![rddpageafter](https://cloud.githubusercontent.com/assets/9278199/8951536/439d66e0-3601-11e5-9cee-1b380fe6620d.png)

Author: Carson Wang <carson.wang@intel.com>

Closes #7692 from carsonwang/SPARK-2016 and squashes the following commits:

03c7168 [Carson Wang] Fix style issues
612c18c [Carson Wang] RDD partition table pagination for the RDD Page

(cherry picked from commit cb7fa0a)
Signed-off-by: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
@asfgit asfgit closed this in cb7fa0a Aug 4, 2015
@carsonwang carsonwang deleted the SPARK-2016 branch August 17, 2015 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@carsonwang @SparkQA @andrewor14 @JoshRosen @sarutak @zsxwing