-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-21254] [WebUI] History UI performance fixes #18783
[SPARK-21254] [WebUI] History UI performance fixes #18783
Commits on Jul 31, 2017
-
[SPARK-21254][WebUI] Improving performance by detaching table DOM bef…
…ore processing Currently all the DOM manipulations are handled in a loop after Mustache template is parsed. This causes severe performance issues especially within loops iteration over thousands of (attempt/application) records and causing all kinds of unnecessary browser work: reflow, repaint, etc. This could be easily fixed by preparing a DOM node beforehand and doing all manipulations within the loops on detached node, reattaching it to the document only after the work is done. The most costly operation in this case was setting innerHTML for `duration` column within a loop, which is extremely unperformant: https://jsperf.com/jquery-append-vs-html-list-performance/24 While duration parsing could be done before mustache-template processing without any additional DOM alteratoins.
Configuration menu - View commit details
-
Copy full SHA for 114943b - Browse repository at this point
Copy the full SHA 114943bView commit details -
[SPARK-21254][WebUI] Performance optimization for pagination check
Check whether to display pagination or not on large data sets (10-20k rows) was taking up to 50ms because it was iterating over all rows. This could be easily done by testing length of array before passing it to mustache.
Configuration menu - View commit details
-
Copy full SHA for 332b398 - Browse repository at this point
Copy the full SHA 332b398View commit details -
[SPARK-21254][WebUI] Removing unnecessary DOM processing
Logic related to `hasMultipleAttempts` flag: - Hiding attmptId column (if `hasMultipleAttempts = false`) - Seting white background color for first 2 columns (if `hasMultipleAttempts = true`) was updating DOM after mustache template processing, which was causing 2 unnecessary iterations over full data set (first through jquery selector, than through for-loop). Refactoring it inside mustache template helps saving 80-90ms on large data sets (10k+ rows)
Configuration menu - View commit details
-
Copy full SHA for 0af596a - Browse repository at this point
Copy the full SHA 0af596aView commit details -
[SPARK-21254][WebUI] further reducing DOM manipulations
Refactoring incomplete requests filter behavior due to inefficency in DOM manipulations. We were traversing DOM multiple times just to hide columns that we could have avoided rendering in mustache (end date, duration). Factoring this logic in mustache template (`showCompletedColumn`) saves 70-80ms on 10k+ rows.
Configuration menu - View commit details
-
Copy full SHA for 235f164 - Browse repository at this point
Copy the full SHA 235f164View commit details -
[SPARK-21254][WebUI] Detaching DOM before DataTables plugin processing
Detaching history table wrapper from document before parsing it with DataTables plugin and reattaching back right after plugin has processed nested DOM. This allows to avoid huge amount of browser repaints and reflows, reducing initial page load time in Chrome from 15s to 4s for 20k+ rows
Configuration menu - View commit details
-
Copy full SHA for f35fdcd - Browse repository at this point
Copy the full SHA f35fdcdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 96598ab - Browse repository at this point
Copy the full SHA 96598abView commit details
Commits on Aug 1, 2017
-
Revert "[HDP-6774] Fixing failing tests by updating HtmlUnit driver d…
…ependency" Upgrade of HtmlUnit seems to bring new dependencies and since it is not required for the purposes of PR (`HistoryServerSuite` tests pass anyway on our Jenkins), we revert HtmlUnit version change This reverts commit 96598ab.
Configuration menu - View commit details
-
Copy full SHA for 4a61ac1 - Browse repository at this point
Copy the full SHA 4a61ac1View commit details