Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-29034][SQL] PostgreSQL dialect conformance for string constants with C-style escapes #26715

Closed
wants to merge 4 commits into from

Conversation

yaooqinn
Copy link
Member

@yaooqinn yaooqinn commented Nov 29, 2019

What changes were proposed in this pull request?

On one hand, we now use spark.sql.parser.escapedStringLiterals to control whether to escape string literal or not.
On the other hand, we use spark.sql.dialect to choose spark or PostgreSQL dialect. When we use the PostgreSQL dialect, we should obey the C-style escape behavior of PostgreSQL.

Supported

An escape string constant is specified by writing the letter E (upper or lower case) just before the opening single quote, e.g., E'foo'.

Not supported

When continuing an escape string constant across lines, write E only before the first opening quote.

Because PostgreSQL follows the SQL standard that is

Two string constants that are only separated by whitespace with at least one newline are concatenated and effectively treated as if the string had been written as one constant.

, which is hard to follow in Spark's Parser.

Why are the changes needed?

PostgreSQL dialect conformance

Does this PR introduce any user-facing change?

yes, when we use the PostgreSQL dialect, we use 'E' to define an escape string constant

How was this patch tested?

add ut.

@yaooqinn
Copy link
Member Author

retest this please

@SparkQA
Copy link

SparkQA commented Nov 29, 2019

Test build #114629 has finished for PR 26715 at commit 7ae10e9.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 29, 2019

Test build #114634 has finished for PR 26715 at commit db93221.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@yaooqinn
Copy link
Member Author

closed because pg dialect removed

@yaooqinn yaooqinn closed this Dec 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants