Skip to content

[SPARK-4648][SQL] Support COALESCE function in Spark SQL and Hive QL#3510

Closed
ravipesala wants to merge 6 commits intoapache:masterfrom
ravipesala:Coalesce
Closed

[SPARK-4648][SQL] Support COALESCE function in Spark SQL and Hive QL#3510
ravipesala wants to merge 6 commits intoapache:masterfrom
ravipesala:Coalesce

Conversation

@ravipesala
Copy link
Contributor

Support Coalesce function in Spark SQL.
Support type widening in Coalesce function.
And replace Coalesce UDF in Spark Hive with local Coalesce function since it is memory efficient and faster.

@SparkQA
Copy link

SparkQA commented Nov 28, 2014

Test build #23938 has started for PR 3510 at commit bbdeebe.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Nov 28, 2014

Test build #23938 has finished for PR 3510 at commit bbdeebe.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23938/
Test FAILed.

@ravipesala ravipesala changed the title [SPARK-4648][SQL] Support COALESCE function in Spark SQL and Hive QL [SPARK-4648][SQL] Support COALESCE function in Spark SQL Nov 28, 2014
@SparkQA
Copy link

SparkQA commented Nov 28, 2014

Test build #23942 has started for PR 3510 at commit 9fa1f9e.

  • This patch merges cleanly.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23940/
Test FAILed.

@ravipesala
Copy link
Contributor Author

retest please

@SparkQA
Copy link

SparkQA commented Nov 28, 2014

Test build #23942 has finished for PR 3510 at commit 9fa1f9e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23942/
Test PASSed.

@SparkQA
Copy link

SparkQA commented Nov 29, 2014

Test build #23949 has started for PR 3510 at commit e377a0c.

  • This patch merges cleanly.

@ravipesala ravipesala changed the title [SPARK-4648][SQL] Support COALESCE function in Spark SQL [SPARK-4648][SQL] Support COALESCE function in Spark SQL and Hive QL Nov 29, 2014
@SparkQA
Copy link

SparkQA commented Nov 29, 2014

Test build #23949 has finished for PR 3510 at commit e377a0c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23949/
Test PASSed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Move that into the pattern matching guard?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though then you would have to do the map/distinct twice I believe.

@chenghao-intel
Copy link
Contributor

That's a good catch, but the children data type converting criteria for Coalesce need to be discussed more deeply. Generally, I think we'd better keep it the same as Hive does. And it will be nicer if a Hive query suite provided.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indentation is off here.

@marmbrus
Copy link
Contributor

Thanks for working on this! In order to keep the PR queue small, I propose we close this issue for now and reopen it once we figure out the correct coercion semantics and add more tests.

@asfgit asfgit closed this in ca12608 Dec 17, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants