Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-29840][SQL] PostgreSQL dialect: cast to integer #26481

Closed
wants to merge 6 commits into from

Conversation

amanomer
Copy link
Contributor

@amanomer amanomer commented Nov 12, 2019

What changes were proposed in this pull request?

To make SparkSQL's cast as int behavior consistent with PostgreSQL when spark.sql.dialect is configured as PostgreSQL.

Why are the changes needed?

SparkSQL and PostgreSQL have a lot different cast behavior between types by default. We should make SparkSQL's cast behavior be consistent with PostgreSQL when spark.sql.dialect is configured as PostgreSQL.

Does this PR introduce any user-facing change?

Yes. If user switches to PostgreSQL dialect now, they will

  • get an AnalysisException when they try to cast ByteType, TimestampType or DateType to IntegerType

How was this patch tested?

Added test cases.

@amanomer amanomer changed the title [WIP][SPARK-29838][SQL] PostgreSQL dialect: cast to integer [WIP][SPARK-29840][SQL] PostgreSQL dialect: cast to integer Nov 12, 2019
@amanomer amanomer changed the title [WIP][SPARK-29840][SQL] PostgreSQL dialect: cast to integer [SPARK-29840][SQL] PostgreSQL dialect: cast to integer Nov 14, 2019
@amanomer
Copy link
Contributor Author

cc @cloud-fan

@amanomer
Copy link
Contributor Author

Retest this please

@maropu
Copy link
Member

maropu commented Nov 15, 2019

memo: pending until #26472 merged.

@dongjoon-hyun
Copy link
Member

Thank you for contribution, @amanomer . As you know, unfortunately, we decided to remove PostgreSQL dialect via SPARK-30125 (#26763). Sorry about that. I'll close this PR, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants