Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-21254] [WebUI] History UI performance fixes #18783

Closed

Commits on Jul 31, 2017

  1. [SPARK-21254][WebUI] Improving performance by detaching table DOM bef…

    …ore processing
    
    Currently all the DOM manipulations are handled in a loop after Mustache
    template is parsed. This causes severe performance issues especially within
    loops iteration over thousands of (attempt/application) records and causing
    all kinds of unnecessary browser work: reflow, repaint, etc.
    
    This could be easily fixed by preparing a DOM node beforehand and doing all
    manipulations within the loops on detached node, reattaching it to the document
    only after the work is done.
    
    The most costly operation in this case was setting innerHTML for `duration`
    column within a loop, which is extremely unperformant:
    
    https://jsperf.com/jquery-append-vs-html-list-performance/24
    
    While duration parsing could be done before mustache-template processing without
    any additional DOM alteratoins.
    2ooom committed Jul 31, 2017
    Configuration menu
    Copy the full SHA
    114943b View commit details
    Browse the repository at this point in the history
  2. [SPARK-21254][WebUI] Performance optimization for pagination check

    Check whether to display pagination or not on large data sets (10-20k rows)
    was taking up to 50ms because it was iterating over all rows. This could be
    easily done by testing length of array before passing it to mustache.
    2ooom committed Jul 31, 2017
    Configuration menu
    Copy the full SHA
    332b398 View commit details
    Browse the repository at this point in the history
  3. [SPARK-21254][WebUI] Removing unnecessary DOM processing

    Logic related to `hasMultipleAttempts` flag:
    
     - Hiding attmptId column (if `hasMultipleAttempts = false`)
     - Seting white background color for first 2 columns (if `hasMultipleAttempts = true`)
    
    was updating DOM after mustache template processing, which was causing 2 unnecessary
    iterations over full data set (first through jquery selector, than through for-loop).
    
    Refactoring it inside mustache template helps saving 80-90ms on large data sets (10k+ rows)
    2ooom committed Jul 31, 2017
    Configuration menu
    Copy the full SHA
    0af596a View commit details
    Browse the repository at this point in the history
  4. [SPARK-21254][WebUI] further reducing DOM manipulations

    Refactoring incomplete requests filter behavior due to inefficency in DOM
    manipulations. We were traversing DOM multiple times just to hide columns
    that we could have avoided rendering in mustache (end date, duration).
    Factoring this logic in mustache template (`showCompletedColumn`) saves
    70-80ms on 10k+ rows.
    2ooom committed Jul 31, 2017
    Configuration menu
    Copy the full SHA
    235f164 View commit details
    Browse the repository at this point in the history
  5. [SPARK-21254][WebUI] Detaching DOM before DataTables plugin processing

    Detaching history table wrapper from document before parsing it with DataTables plugin
    and reattaching back right after plugin has processed nested DOM. This allows to avoid
    huge amount of browser repaints and reflows, reducing initial page load time in Chrome
    from 15s to 4s for 20k+ rows
    2ooom committed Jul 31, 2017
    Configuration menu
    Copy the full SHA
    f35fdcd View commit details
    Browse the repository at this point in the history
  6. [HDP-6774] Fixing failing tests by updating HtmlUnit driver dependency

    Anna Savarin authored and 2ooom committed Jul 31, 2017
    Configuration menu
    Copy the full SHA
    96598ab View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2017

  1. Revert "[HDP-6774] Fixing failing tests by updating HtmlUnit driver d…

    …ependency"
    
    Upgrade of HtmlUnit seems to bring new dependencies and since it is not required
    for the purposes of PR (`HistoryServerSuite` tests pass anyway on our Jenkins),
    we revert HtmlUnit version change
    
    This reverts commit 96598ab.
    2ooom committed Aug 1, 2017
    Configuration menu
    Copy the full SHA
    4a61ac1 View commit details
    Browse the repository at this point in the history