[SPARK-7749] [SQL] Fixes partition discovery for non-partitioned tables #6287

liancheng · 2015-05-20T17:38:29Z

When no partition columns can be found, we should have an empty PartitionSpec, rather than a PartitionSpec with empty partition columns.

This PR together with #6285 should fix SPARK-7749.

AmplabJenkins · 2015-05-20T17:42:12Z

Merged build triggered.

AmplabJenkins · 2015-05-20T17:42:18Z

Merged build started.

SparkQA · 2015-05-20T17:44:19Z

Test build #33151 has started for PR 6287 at commit 19ae41e.

SparkQA · 2015-05-20T18:02:07Z

Test build #33151 has finished for PR 6287 at commit 19ae41e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-20T18:02:11Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-20T18:02:11Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33151/
Test FAILed.

AmplabJenkins · 2015-05-20T18:27:11Z

Merged build triggered.

AmplabJenkins · 2015-05-20T18:27:17Z

Merged build started.

SparkQA · 2015-05-20T18:29:13Z

Test build #33157 has started for PR 6287 at commit fc56656.

SparkQA · 2015-05-20T20:29:01Z

Test build #33157 has finished for PR 6287 at commit fc56656.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-20T20:29:06Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-20T20:29:06Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33157/
Test PASSed.

AmplabJenkins · 2015-05-20T20:52:12Z

Merged build triggered.

AmplabJenkins · 2015-05-20T20:52:19Z

Merged build started.

SparkQA · 2015-05-20T20:53:10Z

Test build #33171 has started for PR 6287 at commit 5aa87ea.

SparkQA · 2015-05-20T22:43:52Z

Test build #33171 has finished for PR 6287 at commit 5aa87ea.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-20T22:43:57Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-20T22:43:57Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33171/
Test PASSed.

AmplabJenkins · 2015-05-21T13:02:11Z

Merged build triggered.

AmplabJenkins · 2015-05-21T13:02:18Z

Merged build started.

SparkQA · 2015-05-21T13:04:15Z

Test build #33245 has started for PR 6287 at commit c4949be.

SparkQA · 2015-05-21T14:57:57Z

Test build #33245 has finished for PR 6287 at commit c4949be.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-21T14:58:02Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-21T14:58:02Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33245/
Test PASSed.

AmplabJenkins · 2015-05-21T15:57:13Z

Merged build triggered.

AmplabJenkins · 2015-05-21T15:57:21Z

Merged build started.

SparkQA · 2015-05-21T15:57:45Z

Test build #33250 has started for PR 6287 at commit a799ff3.

SparkQA · 2015-05-21T17:54:53Z

Test build #33250 has finished for PR 6287 at commit a799ff3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-21T17:54:57Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-21T17:54:57Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33250/
Test PASSed.

yhuai · 2015-05-21T17:55:55Z

LGTM. I am merging it to master and branch 1.4.

When no partition columns can be found, we should have an empty `PartitionSpec`, rather than a `PartitionSpec` with empty partition columns. This PR together with #6285 should fix SPARK-7749. Author: Cheng Lian <lian@databricks.com> Author: Yin Huai <yhuai@databricks.com> Closes #6287 from liancheng/spark-7749 and squashes the following commits: a799ff3 [Cheng Lian] Adds test cases for SPARK-7749 c4949be [Cheng Lian] Minor refactoring, and tolerant _TEMPORARY directory name 5aa87ea [Yin Huai] Make parsePartitions more robust. fc56656 [Cheng Lian] Returns empty PartitionSpec if no partition columns can be inferred 19ae41e [Cheng Lian] Don't list base directory as leaf directory (cherry picked from commit 8730fbb) Signed-off-by: Yin Huai <yhuai@databricks.com>

When no partition columns can be found, we should have an empty `PartitionSpec`, rather than a `PartitionSpec` with empty partition columns. This PR together with apache#6285 should fix SPARK-7749. Author: Cheng Lian <lian@databricks.com> Author: Yin Huai <yhuai@databricks.com> Closes apache#6287 from liancheng/spark-7749 and squashes the following commits: a799ff3 [Cheng Lian] Adds test cases for SPARK-7749 c4949be [Cheng Lian] Minor refactoring, and tolerant _TEMPORARY directory name 5aa87ea [Yin Huai] Make parsePartitions more robust. fc56656 [Cheng Lian] Returns empty PartitionSpec if no partition columns can be inferred 19ae41e [Cheng Lian] Don't list base directory as leaf directory

Don't list base directory as leaf directory

19ae41e

liancheng changed the title ~~[SPARK-7749] [SQL] Fixes partition discovery for non-partitioned tables~~ [SPARK-7749] [SQL] [WIP] Fixes partition discovery for non-partitioned tables May 20, 2015

Returns empty PartitionSpec if no partition columns can be inferred

fc56656

Make parsePartitions more robust.

5aa87ea

Minor refactoring, and tolerant _TEMPORARY directory name

c4949be

Adds test cases for SPARK-7749

a799ff3

liancheng changed the title ~~[SPARK-7749] [SQL] [WIP] Fixes partition discovery for non-partitioned tables~~ [SPARK-7749] [SQL] Fixes partition discovery for non-partitioned tables May 21, 2015

asfgit closed this in 8730fbb May 21, 2015

liancheng deleted the spark-7749 branch May 21, 2015 22:52

[SPARK-7749] [SQL] Fixes partition discovery for non-partitioned tables #6287

[SPARK-7749] [SQL] Fixes partition discovery for non-partitioned tables #6287

Uh oh!

Conversation

liancheng commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

SparkQA commented May 20, 2015

Uh oh!

SparkQA commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

SparkQA commented May 20, 2015

Uh oh!

SparkQA commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

SparkQA commented May 20, 2015

Uh oh!

SparkQA commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 20, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

SparkQA commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

AmplabJenkins commented May 21, 2015

Uh oh!

yhuai commented May 21, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants