Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-28020][SQL][TEST] Port date.sql #24850

Closed
wants to merge 6 commits into from

Conversation

Projects
None yet
4 participants
@wangyum
Copy link
Contributor

commented Jun 12, 2019

What changes were proposed in this pull request?

This PR is to port date.sql from PostgreSQL regression tests. https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/sql/date.sql

The expected results can be found in the link: https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/expected/date.out

When porting the test cases, found four PostgreSQL specific features that do not exist in Spark SQL:

SPARK-28017: Enhance EXTRACT/DATE_TRUNC
SPARK-28141: Date type can not accept special values
SPARK-28253: Date type have different low value and high value
SPARK-28259: Date/Time Output Styles and Date Order Conventions

Also, found a bug:
SPARK-28015: Invalid date formats should throw an exception

Also, found a inconsistent behavior:
SPARK-27923: Invalid date throw an exception bug Spark SQL returns NULL, for example: https://github.com/postgres/postgres/blob/30bcebbdcf23eb8b78e553c4b3b5eb847410ef19/src/test/regress/expected/date.out#L13-L14

How was this patch tested?

N/A

@SparkQA

This comment has been minimized.

Copy link

commented Jun 12, 2019

Test build #106413 has finished for PR 24850 at commit 69470df.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun dongjoon-hyun added TEST SQL TESTS and removed TEST labels Jun 12, 2019

--
--
-- DATE
-- https://github.com/postgres/postgres/blob/REL_12_BETA1/src/test/regress/sql/date.sql

This comment has been minimized.

Copy link
@dongjoon-hyun

dongjoon-hyun Jul 4, 2019

Member

PostgresSQL 12 Beta2 was released on 20th June.

Could you check https://github.com/postgres/postgres/blob/REL_12_BETA1/src/test/regress/sql/date.sql and update this PR (if needed)?

This comment has been minimized.

Copy link
@wangyum

wangyum Jul 4, 2019

Author Contributor

Only these files changed:
image

This comment has been minimized.

Copy link
@dongjoon-hyun

dongjoon-hyun Jul 4, 2019

Member

Thanks. Then, let's update this line 6 from REL_12_BETA1 to REL_12_BETA2 because we had better point to the latest branch.

This comment has been minimized.

Copy link
@maropu

maropu Jul 4, 2019

Member

btw, we don't need to have the same snapshot of these migrated pg tests? In sql-tests/inputs/pgSQL, already-merged tests comes from REL_12_BETA1 and the other tests will come from REL_12_BETA2?

This comment has been minimized.

Copy link
@dongjoon-hyun

dongjoon-hyun Jul 5, 2019

Member

Yes. I think so, @maropu . If we started with REL_11, it might be a different story. However, we don't need to stick to BETA1. It's unofficially one. We just picked up them because that was the latest one at that time.

Also, we can update them later consistently and easily. For now, it seems that the best effort is to keep using the latest one at the merge time.

This comment has been minimized.

Copy link
@maropu

maropu Jul 7, 2019

Member

ok, thanks!

@wangyum wangyum changed the title [WIP][SPARK-28020][SQL][TEST] Port date.sql [SPARK-28020][SQL][TEST] Port date.sql Jul 4, 2019

@SparkQA

This comment has been minimized.

Copy link

commented Jul 5, 2019

Test build #107245 has finished for PR 24850 at commit 06f8bf1.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
@dongjoon-hyun

This comment has been minimized.

Copy link
Member

commented Jul 5, 2019

Only a few comments. Also, please read and update the PR description (e.g. it still mentions BETA1).

@SparkQA

This comment has been minimized.

Copy link

commented Jul 5, 2019

Test build #107282 has finished for PR 24850 at commit 589745d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
@SparkQA

This comment has been minimized.

Copy link

commented Jul 6, 2019

Test build #107293 has finished for PR 24850 at commit 52ccc4f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
@dongjoon-hyun
Copy link
Member

left a comment

+1, LGTM. This is the current status of master branch.
Merged to master.
Thank you, @wangyum and @maropu .

@wangyum wangyum deleted the wangyum:SPARK-28020 branch Jul 6, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.