Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-28326][SQL][TEST] Port join.sql #25148

Closed
wants to merge 6 commits into from
Closed

[SPARK-28326][SQL][TEST] Port join.sql #25148

wants to merge 6 commits into from

Conversation

wangyum
Copy link
Member

@wangyum wangyum commented Jul 14, 2019

What changes were proposed in this pull request?

This PR is to port join.sql from PostgreSQL regression tests. https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/sql/join.sql

The expected results can be found in the link: https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/expected/join.out

When porting the test cases, found nine PostgreSQL specific features that do not exist in Spark SQL:
SPARK-27877: ANSI SQL: LATERAL derived table(T491)
SPARK-20856: support statement using nested joins
SPARK-27987: Support POSIX Regular Expressions
SPARK-28382: Array Functions: unnest
SPARK-25411: Implement range partition in Spark
SPARK-28377: Fully support correlation names in the FROM clause
SPARK-28330: Enhance query limit
SPARK-28379: Correlated scalar subqueries must be aggregated
SPARK-16452: basic INFORMATION_SCHEMA support

How was this patch tested?

N/A

@wangyum wangyum changed the title [SPARK-28326][SQL][TEST] Add join.sql [SPARK-28326][SQL][TEST] Port join.sql Jul 14, 2019
@SparkQA
Copy link

SparkQA commented Jul 14, 2019

Test build #107639 has finished for PR 25148 at commit 3942d87.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jul 14, 2019

Test build #107645 has finished for PR 25148 at commit f329e6d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

Retest this please.

@SparkQA
Copy link

SparkQA commented Jul 28, 2019

Test build #108258 has finished for PR 25148 at commit f329e6d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

From the PR description, please replace numeric.sql with join.sql.

This PR is to port numeric.sql from PostgreSQL regression tests.

@SparkQA
Copy link

SparkQA commented Jul 29, 2019

Test build #108332 has finished for PR 25148 at commit 12ff0b2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@maropu
Copy link
Member

maropu commented Jul 30, 2019

It seems like there is no problem in the output (I checked the answers between Spark and PostgreSQL as much as possible though, its too big to check one-by-one...)

@wangyum
Copy link
Member Author

wangyum commented Jul 30, 2019

Thank you @maropu

@SparkQA
Copy link

SparkQA commented Jul 30, 2019

Test build #108363 has finished for PR 25148 at commit 6261c87.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @wangyum and @maropu .
Merged to master.

@wangyum wangyum deleted the SPARK-28326 branch July 30, 2019 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants