Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-10548] [SPARK-10563] [SQL] Fix concurrent SQL executions #8710

Closed
wants to merge 11 commits into from

Commits on Sep 10, 2015

  1. Exclude certain local properties from being inherited

    such as, cough cough, the SQL execution ID. This was a problem
    because scala's parallel collections spawns threads as children
    of the existing threads, causing the execution ID to be inherited
    when it shouldn't be.
    Andrew Or committed Sep 10, 2015
    Configuration menu
    Copy the full SHA
    8ceae42 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2015

  1. Fix remove from Properties + add tests

    Because java.util.Properties' remove method takes in an Any
    instead of a String, there were some issues with matching the
    key's hashCode, so removing was not successful in unit tests.
    
    Instead, this commit fixes it by manually filtering out the keys
    and adding them to the child thread's properties.
    Andrew Or committed Sep 11, 2015
    Configuration menu
    Copy the full SHA
    3ec715c View commit details
    Browse the repository at this point in the history
  2. Fix style

    Andrew Or committed Sep 11, 2015
    Configuration menu
    Copy the full SHA
    d48c114 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'master' of github.com:apache/spark into concurrent-sql-…

    …executions
    Andrew Or committed Sep 11, 2015
    Configuration menu
    Copy the full SHA
    bbda199 View commit details
    Browse the repository at this point in the history
  4. Always clone parent properties

    ... to make the behavior more consistent in SQL vs non-SQL cases.
    Andrew Or committed Sep 11, 2015
    Configuration menu
    Copy the full SHA
    5297f79 View commit details
    Browse the repository at this point in the history
  5. Merge branch 'master' of github.com:apache/spark into concurrent-sql-…

    …executions
    
    Conflicts:
    	core/src/test/scala/org/apache/spark/ThreadingSuite.scala
    Andrew Or committed Sep 11, 2015
    Configuration menu
    Copy the full SHA
    3c00cc6 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2015

  1. Merge branch 'master' of github.com:apache/spark into concurrent-sql-…

    …executions
    Andrew Or committed Sep 13, 2015
    Configuration menu
    Copy the full SHA
    801fbe0 View commit details
    Browse the repository at this point in the history
  2. Limit scope of mutable set

    Andrew Or committed Sep 13, 2015
    Configuration menu
    Copy the full SHA
    35bb6f0 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2015

  1. Simplify fix by cloning properties on inherit

    The fix for SPARK-10548 can be simplified by just cloning the
    parent properties on inherit rather than excluding specific
    properties from ever being inherited. This is safe because the
    child thread must be created BEFORE the parent thread runs a
    query.
    Andrew Or committed Sep 14, 2015
    Configuration menu
    Copy the full SHA
    984a92f View commit details
    Browse the repository at this point in the history
  2. Merge branch 'master' of github.com:apache/spark into concurrent-sql-…

    …executions
    Andrew Or committed Sep 14, 2015
    Configuration menu
    Copy the full SHA
    fce3819 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'master' of github.com:apache/spark into concurrent-sql-…

    …executions
    
    Conflicts:
    	core/src/test/scala/org/apache/spark/ThreadingSuite.scala
    Andrew Or committed Sep 14, 2015
    Configuration menu
    Copy the full SHA
    75a8d90 View commit details
    Browse the repository at this point in the history