Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-5411] Allow SparkListeners to be specified in SparkConf and loaded when creating SparkContext #4111

Closed
wants to merge 12 commits into from
Closed

[SPARK-5411] Allow SparkListeners to be specified in SparkConf and loaded when creating SparkContext #4111

wants to merge 12 commits into from

Commits on Jan 20, 2015

  1. [SPARK-5190] Allow SparkListeners to be registered before SparkContex… 

    …t starts.
    @JoshRosen
    JoshRosen committed Jan 20, 2015
    Configuration menu
    Copy the full SHA
    163ba19 View commit details
    Browse the repository at this point in the history

  2. Add addSparkListener to JavaSparkContext

    @JoshRosen
    JoshRosen committed Jan 20, 2015
    Configuration menu
    Copy the full SHA
    25988f3 View commit details
    Browse the repository at this point in the history

  3. Revert "Add addSparkListener to JavaSparkContext" 

    This reverts commit 25988f3.
    @JoshRosen
    JoshRosen committed Jan 20, 2015
    Configuration menu
    Copy the full SHA
    217ecc0 View commit details
    Browse the repository at this point in the history

  4. Revert "[SPARK-5190] Allow SparkListeners to be registered before Spa… 

    …rkContext starts."

This reverts commit 163ba19.
    @JoshRosen
    JoshRosen committed Jan 20, 2015
    Configuration menu
    Copy the full SHA
    9c0d8f1 View commit details
    Browse the repository at this point in the history

  5. Instantiate SparkListeners from classes listed in configurations.

    @JoshRosen
    JoshRosen committed Jan 20, 2015
    Configuration menu
    Copy the full SHA
    b22b379 View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2015

  1. Move code into setupAndStartListenerBus() method

    @JoshRosen
    JoshRosen committed Jan 26, 2015
    Configuration menu
    Copy the full SHA
    d0d276d View commit details
    Browse the repository at this point in the history

  2. Use getConstructors() instead of try-catch to find right constructor.

    @JoshRosen
    JoshRosen committed Jan 26, 2015
    Configuration menu
    Copy the full SHA
    d6f3113 View commit details
    Browse the repository at this point in the history

  3. Add test to ensure that conf and env var settings are merged, not ove… 

    …rriden.
    @JoshRosen
    JoshRosen committed Jan 26, 2015
    Configuration menu
    Copy the full SHA
    b9973da View commit details
    Browse the repository at this point in the history

  4. Add a couple of explanatory comments for SPARK_EXTRA_LISTENERS.

    @JoshRosen
    JoshRosen committed Jan 26, 2015
    Configuration menu
    Copy the full SHA
    2daff9b View commit details
    Browse the repository at this point in the history

  5. Remove SPARK_EXTRA_LISTENERS environment variable.

    @JoshRosen
    JoshRosen committed Jan 26, 2015
    Configuration menu
    Copy the full SHA
    1a5b9a0 View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2015

  1. Merge remote-tracking branch 'origin/master' into SPARK-5190-register… 

    …-sparklistener-in-sc-constructor

Conflicts:
	core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala
	core/src/main/scala/org/apache/spark/scheduler/SparkListenerBus.scala
    @JoshRosen
    JoshRosen committed Feb 2, 2015
    Configuration menu
    Copy the full SHA
    6e0122c View commit details
    Browse the repository at this point in the history

  2. Two minor fixes after merging with master

    @JoshRosen
    JoshRosen committed Feb 2, 2015
    Configuration menu
    Copy the full SHA
    8370839 View commit details
    Browse the repository at this point in the history