[SPARK-9596][SQL]treat hadoop classes as shared one in IsolatedClientLoader

[WangTaoTheTonic]

https://issues.apache.org/jira/browse/SPARK-9596

[WangTaoTheTonic]

Jenkins, test this please.

[SparkQA]

Test build #210 has finished for PR 7931 at commit 3abe66c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #39712 has finished for PR 7931 at commit 3abe66c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #39724 has finished for PR 7931 at commit 3abe66c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[WangTaoTheTonic]

The error is:

stderr> FAILED: SemanticException [Error 10072]: Database does not exist: hive_test_db
[info] stderr> Exception in thread "main" org.apache.spark.sql.execution.QueryExecutionException: FAILED: SemanticException [Error 10072]: Database does not exist: hive_test_db

I should not be produced by this patch.

Jenkins, retest this please.

[WangTaoTheTonic]

Jenkins, retest this please.

[SparkQA]

Test build #221 has finished for PR 7931 at commit 3abe66c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #39797 has finished for PR 7931 at commit 3abe66c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[WangTaoTheTonic]

Jenkins, retest this please.

[SparkQA]

Test build #226 has finished for PR 7931 at commit 3abe66c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #39837 has finished for PR 7931 at commit 3abe66c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #39855 has finished for PR 7931 at commit cdfbe24.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[WangTaoTheTonic]

@marmbrus I am not sure why it will leads to a test failed. Could you help to check this patch and the failed reason? Thanks.

[marmbrus]

It looks like your patch is breaking Hive.

[marmbrus]

In particular I think that some of hives classes exist in the hadoop namespace so cannot be shared.

[SparkQA]

Test build #39973 has finished for PR 7931 at commit 42ea1b5.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[WangTaoTheTonic]

@marmbrus Looks like the patch is ok with excluding hive classes. Thanks for your guide :)

[marmbrus]

Thanks, merged to master.

[Sephiroth-Lin]

scope is too huge, and without reload hadoop, will only have one FileSystem.Cache which will cause SPARK-11083