Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-19598][SQL]Remove the alias parameter in UnresolvedRelation #16956

Closed
wants to merge 6 commits into from

Conversation

windpiger
Copy link
Contributor

What changes were proposed in this pull request?

Remove the alias parameter in UnresolvedRelation, and use SubqueryAlias to replace it.
This can simplify some match case situations.

For example, the broadcast hint pull request can have one fewer case https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala#L57-L61

How was this patch tested?

add some unit tests

@@ -54,10 +54,6 @@ object ResolveHints {

val newNode = CurrentOrigin.withOrigin(plan.origin) {
plan match {
case r: UnresolvedRelation =>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually you can't remove this case entirely. you still need to match on UnresolvedRelation, but just use r.tableIdentifier.table

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, yes, remove the case will miss the hint on table without alias

@SparkQA
Copy link

SparkQA commented Feb 16, 2017

Test build #72995 has finished for PR 16956 at commit 4023727.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Feb 16, 2017

Test build #72996 has finished for PR 16956 at commit 3c4be76.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Feb 16, 2017

Test build #72999 has finished for PR 16956 at commit accd3b9.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gatorsmile
Copy link
Member

I like this change.

To run unit test cases in your local environment, below are some common commands when I develop Spark SQL

  • To do the local style check
    dev/lint-scala

  • To run the SQL test cases in your local laptop:
    build/sbt -Phive hive/test
    build/sbt sql/test
    build/sbt catalyst/test

val table = UnresolvedRelation(
visitTableIdentifier(ctx.tableIdentifier),
Option(ctx.strictIdentifier).map(_.getText))
table.optionalMap(ctx.sample)(withSample)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be kept in the last line

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I can not modify the original default expression order between alias and and sample.

@windpiger
Copy link
Contributor Author

@gatorsmile thanks for your good suggestion~Usually I run the single test case in IDEA,and let jenkins run testcases globaly,I will change the bad behavior, thanks a lot!

@SparkQA
Copy link

SparkQA commented Feb 17, 2017

Test build #73044 has started for PR 16956 at commit d2864b6.

@windpiger
Copy link
Contributor Author

retest this please

@SparkQA
Copy link

SparkQA commented Feb 17, 2017

Test build #73045 has finished for PR 16956 at commit d2864b6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@windpiger
Copy link
Contributor Author

@rxin @gatorsmile help to continue to review this? Thanks :)

@gatorsmile
Copy link
Member

LGTM

@gatorsmile
Copy link
Member

retest this please

@SparkQA
Copy link

SparkQA commented Feb 19, 2017

Test build #73129 has finished for PR 16956 at commit d2864b6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gatorsmile
Copy link
Member

Thanks! Merging to master.

@asfgit asfgit closed this in 65fe902 Feb 20, 2017
Yunni pushed a commit to Yunni/spark that referenced this pull request Feb 27, 2017
## What changes were proposed in this pull request?

Remove the alias parameter in `UnresolvedRelation`, and use `SubqueryAlias` to replace it.
This can simplify some `match case` situations.

For example, the broadcast hint pull request can have one fewer case https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala#L57-L61

## How was this patch tested?
add some unit tests

Author: windpiger <songjun@outlook.com>

Closes apache#16956 from windpiger/removeUnresolveTableAlias.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants