Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-22537][core] Aggregation of map output statistics on driver faces single point bottleneck #19763

Closed
wants to merge 13 commits into from
Closed

[SPARK-22537][core] Aggregation of map output statistics on driver faces single point bottleneck #19763

wants to merge 13 commits into from

Commits on Nov 16, 2017

  1. Use multi-thread to solve single point bottleneck

    GuoChenzhao committed Nov 16, 2017
    Configuration menu
    Copy the full SHA
    5dd0487 View commit details
    Browse the repository at this point in the history

  2. Add test case

    GuoChenzhao committed Nov 16, 2017
    Configuration menu
    Copy the full SHA
    819774f View commit details
    Browse the repository at this point in the history

  3. Style

    GuoChenzhao committed Nov 16, 2017
    Configuration menu
    Copy the full SHA
    da02825 View commit details
    Browse the repository at this point in the history

  4. Use Scala Future

    GuoChenzhao committed Nov 16, 2017
    Configuration menu
    Copy the full SHA
    2735b30 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2017

  1. Not use global executionContext and add a conf to decide use multi-th… 

    …read or not
    GuoChenzhao committed Nov 20, 2017
    Configuration menu
    Copy the full SHA
    8501970 View commit details
    Browse the repository at this point in the history

  2. New conf name

    GuoChenzhao committed Nov 20, 2017
    Configuration menu
    Copy the full SHA
    4dafb19 View commit details
    Browse the repository at this point in the history

  3. Typo

    GuoChenzhao committed Nov 20, 2017
    Configuration menu
    Copy the full SHA
    3419dfa View commit details
    Browse the repository at this point in the history

  4. Change conf default value, add conf

    GuoChenzhao committed Nov 20, 2017
    Configuration menu
    Copy the full SHA
    da147d7 View commit details
    Browse the repository at this point in the history

  5. Refactor

    GuoChenzhao committed Nov 20, 2017
    Configuration menu
    Copy the full SHA
    8a1719d View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2017

  1. Remove introduced config parallelism, fix step=0 bug and refactor

    GuoChenzhao committed Nov 21, 2017
    Configuration menu
    Copy the full SHA
    055d44c View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2017

  1. Use long to prevent overflow

    @gcz2022
    gcz2022 committed Nov 23, 2017
    Configuration menu
    Copy the full SHA
    72c3d97 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2017

  1. Fix

    GuoChenzhao committed Nov 24, 2017
    Configuration menu
    Copy the full SHA
    c9c26ce View commit details
    Browse the repository at this point in the history

  2. Add more docs and value check

    GuoChenzhao committed Nov 24, 2017
    Configuration menu
    Copy the full SHA
    0f87dd6 View commit details
    Browse the repository at this point in the history