[SPARK-5941] [SQL] Unit Test loads the table src twice for leftsemijoin.q

[chenghao-intel]

In leftsemijoin.q, there is a data loading command for table sales already, but in TestHive, it also created the table sales, which causes duplicated records inserted into the sales.

[SparkQA]

Test build #27207 has finished for PR 4506 at commit 3ee839f.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[marmbrus]

We have eagerly resolved the table since Spark 1.0 when Spark SQL was added.

https://github.com/apache/spark/blob/branch-1.0/sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala#L185

Why do you think this is problematic?

[chenghao-intel]

The deferred resolving table should be harmless, and since most of the DF API are yielding the unresolved logical plans(can I say that?), I think we'd better keep it the same for def table(name), .

Ideally, eagerly resolving the table should produce the the result in unit test, but seems not, probably something wrong somewhere, I will keep investigating that.

[SparkQA]

Test build #27505 has finished for PR 4506 at commit 40ccd81.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #27625 has finished for PR 4506 at commit 4c58fbc.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[chenghao-intel]

/cc @marmbrus @rxin @yhuai I noticed that we have added the dataFrameEagerAnalysis in SQLConf, probably we also need to update the def table code for using UnresolvedRelation instead of lookupRelation eagerly.
However some of the unit test (left semi join) will fail if we do that, as we have bug in TestHive, this PR is also fix the bug. More detailed information about the unit test failure can be found at the description.

[SparkQA]

Test build #27672 has finished for PR 4506 at commit 28dd6d5.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[chenghao-intel]

@marmbrus @rxin @yhuai any more comment? Or should I close this PR?

[rxin]

@marmbrus can you take a look? not 100% sure what's happening

[yhuai]

If I remember it correctly, the issue is some test tables are loaded twice. @chenghao-intel Can you change the title?

[chenghao-intel]

Sorry for the confusing. I've updated the title and description.

[yhuai]

Since sales is not a preloaded in Hive's unit tests (https://github.com/apache/hive/blob/trunk/data/scripts/q_test_init.sql), seems it is fine to remove it.

[SparkQA]

Test build #29735 has finished for PR 4506 at commit b463f8a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.
• This patch does not change any dependencies.

[chenghao-intel]

Closing it since it's only impact a single hive compatible test.

[chenghao-intel]

@marmbrus I've reopened this PR, just in case people runs into the bug of this while unit testing.

[SparkQA]

Test build #30132 has finished for PR 4506 at commit dd0b3f6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.
• This patch does not change any dependencies.

[SparkQA]

Test build #30133 has finished for PR 4506 at commit 0be05f7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.
• This patch does not change any dependencies.

[SparkQA]

Test build #30131 has finished for PR 4506 at commit b463f8a.

• This patch passes all tests.
• This patch does not merge cleanly.
• This patch adds no public classes.

[marmbrus]

Thanks! Merged to master.