Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-7786][STREAMING] Allow StreamingListener to be specified in SparkConf and loaded when creating StreamingContext #6380

Closed
wants to merge 21 commits into from
Closed

[SPARK-7786][STREAMING] Allow StreamingListener to be specified in SparkConf and loaded when creating StreamingContext #6380

wants to merge 21 commits into from

Commits on May 21, 2015

  1. Add func setupStreamingListeners to add listeners from 

    spark.streamingListeners
    @Jeffrharr
    Jeffrharr committed May 21, 2015
    Configuration menu
    Copy the full SHA
    73549ff View commit details
    Browse the repository at this point in the history

Commits on May 23, 2015

  1. Merge branch 'master' into SparkConf_Listeners

    @Jeffrharr
    Jeffrharr committed May 23, 2015
    Configuration menu
    Copy the full SHA
    4eb6987 View commit details
    Browse the repository at this point in the history

  2. Use class API to add listener. Modify StatsReportListener to have a zero 

    argument constructor in order to make it compatible with config option.
    @Jeffrharr
    Jeffrharr committed May 23, 2015
    Configuration menu
    Copy the full SHA
    f8ad629 View commit details
    Browse the repository at this point in the history

  3. Reduced ambiguity by removing default values from StatsReportListener… 

    …'s single-valued

constructor. Added docs for config option spark.streamingListeners
    @Jeffrharr
    Jeffrharr committed May 23, 2015
    Configuration menu
    Copy the full SHA
    3c1a19d View commit details
    Browse the repository at this point in the history

  4. Add test

    @Jeffrharr
    Jeffrharr committed May 23, 2015
    Configuration menu
    Copy the full SHA
    f3a3fee View commit details
    Browse the repository at this point in the history

  5. Fix minor syntax derp

    @Jeffrharr
    Jeffrharr committed May 23, 2015
    Configuration menu
    Copy the full SHA
    e8506d4 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2015

  1. Fix style issues and small type in StreamingListenerSuite

    @Jeffrharr
    Jeffrharr committed May 24, 2015
    Configuration menu
    Copy the full SHA
    6453c90 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2015

  1. Maintain consistancy with property naming conventions

    @Jeffrharr
    Jeffrharr committed May 27, 2015
    Configuration menu
    Copy the full SHA
    d92d55b View commit details
    Browse the repository at this point in the history

  2. Fix test for new format

    @Jeffrharr
    Jeffrharr committed May 27, 2015
    Configuration menu
    Copy the full SHA
    00c0409 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2015

  1. Merge remote-tracking branch 'upstream/master' into SparkConf_Listeners

    @Jeffrharr
    Jeffrharr committed May 28, 2015
    Configuration menu
    Copy the full SHA
    186766f View commit details
    Browse the repository at this point in the history

  2. Test renamed to spark.streaming.listeners

    @Jeffrharr
    Jeffrharr committed May 28, 2015
    Configuration menu
    Copy the full SHA
    c94982f View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2015

  1. Merge remote-tracking branch 'upstream/master' into SparkConf_Listeners

    @Jeffrharr
    Jeffrharr committed Jul 10, 2015
    Configuration menu
    Copy the full SHA
    40d91ed View commit details
    Browse the repository at this point in the history

  2. Update name to spark.streaming.extraListeners / fix docs / change 

    Class.forName to org.apache.spark.util.Utils.classForName
    @Jeffrharr
    Jeffrharr committed Jul 10, 2015
    Configuration menu
    Copy the full SHA
    d7e7b2e View commit details
    Browse the repository at this point in the history

  3. Default to 100

    @Jeffrharr
    Jeffrharr committed Jul 10, 2015
    Configuration menu
    Copy the full SHA
    33d3179 View commit details
    Browse the repository at this point in the history

  4. Update Class.forName to org.apache.spark.util.Utils.classForName to 

    avoid problems with classloaders
    @Jeffrharr
    Jeffrharr committed Jul 10, 2015
    Configuration menu
    Copy the full SHA
    233335d View commit details
    Browse the repository at this point in the history

  5. Use spark.streaming.extraListeners

    @Jeffrharr
    Jeffrharr committed Jul 10, 2015
    Configuration menu
    Copy the full SHA
    28e7f27 View commit details
    Browse the repository at this point in the history

  6. checking

    @Jeffrharr
    Jeffrharr committed Jul 10, 2015
    Configuration menu
    Copy the full SHA
    af41098 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2015

  1. Changes in clarity, removed redundant error handling

    @Jeffrharr
    Jeffrharr committed Jul 13, 2015
    Configuration menu
    Copy the full SHA
    9ec68c1 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2015

  1. Fix documentation, add more/clearer test cases

    @Jeffrharr
    Jeffrharr committed Jul 16, 2015
    Configuration menu
    Copy the full SHA
    70c31e4 View commit details
    Browse the repository at this point in the history

  2. Accidental insertion removed

    @Jeffrharr
    Jeffrharr committed Jul 16, 2015
    Configuration menu
    Copy the full SHA
    aff08ff View commit details
    Browse the repository at this point in the history

  3. Correct test text

    @Jeffrharr
    Jeffrharr committed Jul 16, 2015
    Configuration menu
    Copy the full SHA
    fa8c752 View commit details
    Browse the repository at this point in the history