[SPARK-3675][SQL] Allow starting a JDBC server on an existing context

[marmbrus]

No description provided.

[SparkQA]

QA tests have started for PR 2515 at commit 7866fad.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2515 at commit 7866fad.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20744/

[rxin]

Merging in master. Thanks.

[pwendell]

Is this actually usable in this form? IIRC there are a bunch of e.g. command line arguments the JDBC server takes in order to initialize. What about having a form where you can also pass in an arguments array in addition to a context.

[marmbrus]

@pwendell yeah, sorry I probably should have labeled this WIP, but it does actually work in this form. You just can't configure things like the port to listen on. Unfortunately I think those are read from env vars and so can't be set at runtime anyway...

[chernetsov]

@marmbrus Hi, Michael.

I am trying to make use of the commit you introduced (access my rdd exposed to sqlcontext via registerTempTable) but have problems with it:

• I cherry-picked the change to 1.1.0 release
• edited my spark app to call HiveThriftServer2.startWithContext(sqlContext) and added app thread wait() in the very end so that job does not finish
• launched my app via spark-submit on local master
• all the programmatical sql operations (like sqlContext.sql("select * from ...").collect().foreach(println)) work fine
• thift server starts as well
• but connecting via beeline shows that no tables are exposed to it

0: jdbc:hive2://localhost:10000> show tables;                         
+---------+
| result  |
+---------+
+---------+
No rows selected (0.136 seconds)

Is there any other trick I should do?

[marmbrus]

Temporary tables don't show up in show tables (we should fix this, feel free to open a JIRA). You should still be able to query it though.

[chernetsov]

Thank you for quick answer, @marmbrus
Indeed I was able to query table.

Created the jira to fix show tables: https://issues.apache.org/jira/browse/SPARK-4865