Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-19540][SQL] Add ability to clone SparkSession wherein cloned session has an identical copy of the SessionState #16826

Closed
wants to merge 39 commits into from
Closed

[SPARK-19540][SQL] Add ability to clone SparkSession wherein cloned session has an identical copy of the SessionState #16826

wants to merge 39 commits into from

Commits on Feb 3, 2017

  1. Add capability to inherit SessionState (SQL conf, temp tables, regist… 

    …ered functions) when forking a new SparkSession.
    @kunalkhamar
    kunalkhamar committed Feb 3, 2017
    Configuration menu
    Copy the full SHA
    18ce1b8 View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2017

  1. Add tests for forking new session with inherit config enabled. Update… 

    … overloaded functions for Java bytecode compatibility.
    @kunalkhamar
    kunalkhamar committed Feb 6, 2017
    Configuration menu
    Copy the full SHA
    9beb78d View commit details
    Browse the repository at this point in the history

  2. Fix constructor default args for bytecode compatibility.

    @kunalkhamar
    kunalkhamar committed Feb 6, 2017
    Configuration menu
    Copy the full SHA
    a343d8a View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2017

  1. Incorporate feedback. Fix association of incorrect SparkSession while… 

    … cloning SessionState.
    @kunalkhamar
    kunalkhamar committed Feb 10, 2017
    Configuration menu
    Copy the full SHA
    4210079 View commit details
    Browse the repository at this point in the history

  2. Update spark version. Rename clone to copy, in order to avoid Java Ob… 

    …ject.clone() issues.
    @kunalkhamar
    kunalkhamar committed Feb 10, 2017
    Configuration menu
    Copy the full SHA
    6da6bda View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2017

  1. Make lazy vals strict.

    @kunalkhamar
    kunalkhamar committed Feb 14, 2017
    Configuration menu
    Copy the full SHA
    579d0b7 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2017

  1. Refactor SessionState to remove passing of base SessionState, and ini… 

    …tialize all fields directly instead. Same change for HiveSessionState.
    @kunalkhamar
    kunalkhamar committed Feb 16, 2017
    Configuration menu
    Copy the full SHA
    2837e73 View commit details
    Browse the repository at this point in the history

  2. Remove unused import.

    @kunalkhamar
    kunalkhamar committed Feb 16, 2017
    Configuration menu
    Copy the full SHA
    8c00344 View commit details
    Browse the repository at this point in the history

  3. Remove SparkSession reference from SessionState.

    @kunalkhamar
    kunalkhamar committed Feb 16, 2017
    Configuration menu
    Copy the full SHA
    f423f74 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into fork-sparksession

    @kunalkhamar
    kunalkhamar committed Feb 16, 2017
    Configuration menu
    Copy the full SHA
    b1371d8 View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2017

  1. Fix initialization loop.

    @kunalkhamar
    kunalkhamar committed Feb 17, 2017
    Configuration menu
    Copy the full SHA
    2cee190 View commit details
    Browse the repository at this point in the history

  2. Fix var name error.

    @kunalkhamar
    kunalkhamar committed Feb 17, 2017
    Configuration menu
    Copy the full SHA
    e2bbfa8 View commit details
    Browse the repository at this point in the history

Commits on Feb 18, 2017

  1. Add tests. Refactor. Temporarily disable subtest SPARK-18360: default… 

    … table path of tables in default database should depend on the location of the default database.
    @kunalkhamar
    kunalkhamar committed Feb 18, 2017
    Configuration menu
    Copy the full SHA
    8ac778a View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2017

  1. Merge branch 'master' into fork-sparksession

    @kunalkhamar
    kunalkhamar committed Feb 21, 2017
    Configuration menu
    Copy the full SHA
    0c732ce View commit details
    Browse the repository at this point in the history

  2. Fix copy of SessionCatalog. Changes from review.

    @kunalkhamar
    kunalkhamar committed Feb 21, 2017
    Configuration menu
    Copy the full SHA
    3c995e1 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'fork-sparksession' of github.com:kunalkhamar/spark into… 

    … fork-sparksession

Conflicts:
	sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
    @kunalkhamar
    kunalkhamar committed Feb 21, 2017
    Configuration menu
    Copy the full SHA
    292011a View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into fork-sparksession

    @kunalkhamar
    kunalkhamar committed Feb 21, 2017
    Configuration menu
    Copy the full SHA
    b027412 View commit details
    Browse the repository at this point in the history

  5. Add synchronized blocks. Ignore hive metastore tests for now.

    @kunalkhamar
    kunalkhamar committed Feb 21, 2017
    Configuration menu
    Copy the full SHA
    295ee41 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'fork-sparksession' of github.com:kunalkhamar/spark into… 

    … fork-sparksession
    @kunalkhamar
    kunalkhamar committed Feb 21, 2017
    Configuration menu
    Copy the full SHA
    847b484 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2017

  1. Add tests. Force copy of session state on cloneSession.

    @kunalkhamar
    kunalkhamar committed Feb 22, 2017
    Configuration menu
    Copy the full SHA
    9beba84 View commit details
    Browse the repository at this point in the history

  2. Rename copy to clone() to work around copy method of case classes. Mo… 

    …ve addJar from SessionState to SharedState.
    @kunalkhamar
    kunalkhamar committed Feb 22, 2017
    Configuration menu
    Copy the full SHA
    3d2e4a6 View commit details
    Browse the repository at this point in the history

  3. Fix HiveSessionState clone.

    @kunalkhamar
    kunalkhamar committed Feb 22, 2017
    Configuration menu
    Copy the full SHA
    4f70d12 View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2017

  1. Add tests for HiveSessionState. Review feedback.

    @kunalkhamar
    kunalkhamar committed Feb 23, 2017
    Configuration menu
    Copy the full SHA
    dd2dedd View commit details
    Browse the repository at this point in the history

  2. Simplify TestSQLContext. Review feedback.

    @kunalkhamar
    kunalkhamar committed Feb 23, 2017
    Configuration menu
    Copy the full SHA
    8a8d47b View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2017

  1. (attempt to) Fix tests.

    @kunalkhamar
    kunalkhamar committed Feb 24, 2017
    Configuration menu
    Copy the full SHA
    ffc2058 View commit details
    Browse the repository at this point in the history

  2. Review.

    @kunalkhamar
    kunalkhamar committed Feb 24, 2017
    Configuration menu
    Copy the full SHA
    16824f9 View commit details
    Browse the repository at this point in the history

  3. Update test case.

    @kunalkhamar
    kunalkhamar committed Feb 24, 2017
    Configuration menu
    Copy the full SHA
    fd11ee2 View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2017

  1. Add throwing exception if wrong SessionState clone is called. Update … 

    …tests.
    @kunalkhamar
    kunalkhamar committed Feb 25, 2017
    Configuration menu
    Copy the full SHA
    437b0bc View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2017

  1. Most of the changes from review.

    @kunalkhamar
    kunalkhamar committed Feb 28, 2017
    Configuration menu
    Copy the full SHA
    300d3a0 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2017

  1. All but one review feedback.

    @kunalkhamar
    kunalkhamar committed Mar 6, 2017
    Configuration menu
    Copy the full SHA
    3ee271f View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/master' into pr16826

    @zsxwing
    zsxwing committed Mar 6, 2017
    Configuration menu
    Copy the full SHA
    c3f052f View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/master' into pr16826

    @zsxwing
    zsxwing committed Mar 6, 2017
    Configuration menu
    Copy the full SHA
    0bdc81c View commit details
    Browse the repository at this point in the history

  4. Fix tests

    @zsxwing
    zsxwing committed Mar 6, 2017
    Configuration menu
    Copy the full SHA
    2740c63 View commit details
    Browse the repository at this point in the history

  5. Clean up tests

    @zsxwing
    zsxwing committed Mar 6, 2017
    Configuration menu
    Copy the full SHA
    0f167db View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2017

  1. fix SessionCatalogSuite

    @zsxwing
    zsxwing committed Mar 7, 2017
    Configuration menu
    Copy the full SHA
    2f0b1ad View commit details
    Browse the repository at this point in the history

  2. More cleanup

    @zsxwing
    zsxwing committed Mar 7, 2017
    Configuration menu
    Copy the full SHA
    c41e7bc View commit details
    Browse the repository at this point in the history

  3. More tests

    @zsxwing
    zsxwing committed Mar 7, 2017
    Configuration menu
    Copy the full SHA
    5eb6733 View commit details
    Browse the repository at this point in the history

  4. Update tests and a param comment.

    @kunalkhamar
    kunalkhamar committed Mar 7, 2017
    Configuration menu
    Copy the full SHA
    05abcf8 View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2017

  1. Merge branch 'master' into fork-sparksession

    @kunalkhamar
    kunalkhamar committed Mar 8, 2017
    Configuration menu
    Copy the full SHA
    4c23e7a View commit details
    Browse the repository at this point in the history