[SPARK-30230][SQL] Like ESCAPE syntax can not use '_' and '%'

[ulysses-you]

What changes were proposed in this pull request?

Since 25001, spark support like escape syntax.
But '%' and '_' is the reserve char in Like expression. We can not use them as escape char.

Why are the changes needed?

Avoid some unexpect problem when using like escape syntax.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Add UT.

[ulysses-you]

cc @beliefer @cloud-fan @maropu @gengliangwang

[maropu]

We cannot avoid this case in the antlr side (SqlBase.g4)?

[cloud-fan]

no, the antlr side just mach a string literal.

[ulysses-you]

You mean like this ?

ESCAPE_STRING
    : '\'' ( ~('\''|'\\') | ('\\' .) | ~('%' | '_') )* '\''
    | '\'' ( ~('\''|'\\') | ('\\' .) | ~('%' | '_') )* '\''
    ;

NOT? kind=LIKE pattern=valueExpression (ESCAPE escapeChar=ESCAPE_STRING)?

[cloud-fan]

seems much easier to do at scala side...

[cloud-fan]

with better error message.

[ulysses-you]

Yes, I do agree.

[maropu]

ok.

[cloud-fan]

ok to test

[cloud-fan]

ok to test

[cloud-fan]

add to whitelist

[ulysses-you]

Sorry for a mistake.

[cloud-fan]

nit: don't use Set for just 2 elements. We can just do a = % || a = _

[ulysses-you]

Done.

[maropu]

LGTM except for the @cloud-fan comment.

[SparkQA]

Test build #115224 has finished for PR 26860 at commit 20532d2.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #115236 has finished for PR 26860 at commit 3def8ff.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gengliangwang]

Thanks, merging to master

[ulysses-you]

Thanks.

[beliefer]

@ulysses-you @maropu @cloud-fan @gengliangwang
I checked postgreSQL, it supports as below:
SELECT 'abC' LIKE 'ab%C' ESCAPE '%';

[gengliangwang]

@beliefer you are right. I run the following in PostgreSQL:

> SELECT 'abcd' LIKE 'ab%_d';
true
> SELECT 'abcd' LIKE 'ab%_d' escape '%';
false

I should not merge it without checking on PostgreSQL. I am going to revert this one.
Any thoughts? @ulysses-you @maropu @cloud-fan

[cloud-fan]

Can we check more systems? it does look weird to support % and _ as escape chars.

[ulysses-you]

If '%' and '_' as escape, how to use the real pattern.
Like this ?

SELECT 'abcd' LIKE 'ab%%' escape '%';

But actually StringUtils.escapeLikeRegex will do this check. It can not run correctly.

while (in.hasNext) {
      in.next match {
        case c1 if c1 == escapeChar && in.hasNext =>
          val c = in.next
          c match {
            case '_' | '%' => out ++= Pattern.quote(Character.toString(c))
            case c if c == escapeChar => out ++= Pattern.quote(Character.toString(c))
            case _ => fail(s"the escape character is not allowed to precede '$c'")
          }
        case c if c == escapeChar => fail("it is not allowed to end with the escape character")
        // we will never run these code path
        case '_' => out ++= "."
        case '%' => out ++= ".*"
        case c => out ++= Pattern.quote(Character.toString(c))
      }
    }

[beliefer]

>SELECT 'abcd' LIKE 'ab%%';
true
>SELECT 'abcd' LIKE 'ab%%' ESCAPE '%';
false

Because '%' is escaped.

[cloud-fan]

@gengliangwang let's revert this.

[maropu]

+1 for the revert, thanks for taking care, @gengliangwang !

[ulysses-you]

OK, I see. Thanks for reverting.