Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
c9010a9
Parser and ExpressionBuilder changes for like parameters
cimequinox Nov 11, 2025
21b1e33
Documentation examples for like parameters
cimequinox Nov 13, 2025
4533f2f
Unit tests for like parameters
cimequinox Nov 13, 2025
9dd80b9
Add EsqlCapabilities LIKE_PARAMETER_SUPPORT
cimequinox Nov 13, 2025
61155c7
Add yamlRestTest for like with parameters
cimequinox Nov 13, 2025
52b661f
Generated files
cimequinox Nov 13, 2025
82ec663
Tests check for LIKE_PARAMETER_SUPPORT capability
cimequinox Nov 13, 2025
cb44603
[CI] Auto commit changes from spotless
Nov 13, 2025
b4b732e
Handle RLIKE param, LIKE param list, RLIKE param list
cimequinox Nov 14, 2025
dddd2cc
StatementParserTests for RLIKE, LIKE list, RLIKE list
cimequinox Nov 14, 2025
adfca6b
AnalyzerTests for LIKE, LIKE list, RLIKE, RLIKE list
cimequinox Nov 14, 2025
be2f8b1
Add yamlRestTest for LIKE, RLIKE and error case for LIKE list
cimequinox Nov 14, 2025
0402aca
Add RestEsqlTestCase for LIKE, RLIKE and LIKE list
cimequinox Nov 14, 2025
ff05c50
Fix typo in docs and explain why we use an inline example instead of …
cimequinox Nov 14, 2025
15af547
Generated files
cimequinox Nov 14, 2025
b284375
Update docs/changelog/138051.yaml
cimequinox Nov 17, 2025
ac4d107
Include anonymous and positional parameters in tests
cimequinox Nov 18, 2025
2a065a8
Collect parameter errors instead of immediately throwing them
cimequinox Nov 18, 2025
14a1592
Use anonymous and positional parameters in RestEsqlTestCase and also …
cimequinox Nov 19, 2025
cd652ec
[CI] Auto commit changes from spotless
Nov 19, 2025
d4f112f
Update x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/…
cimequinox Nov 19, 2025
e0b2320
Update x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/…
cimequinox Nov 19, 2025
495fd39
Move stringOrParameter to EsqlBaseParser.g4
cimequinox Nov 19, 2025
d9dbc7f
Generated Files
cimequinox Nov 19, 2025
09bca88
Generated docs
cimequinox Nov 19, 2025
df170f2
Refactor stringToStringOrParam to reuse visitParam via a private help…
cimequinox Nov 19, 2025
53573e3
Merge branch 'main' into esql_like_parameters_131356
cimequinox Nov 19, 2025
7e07be0
Merge branch 'main' into esql_like_parameters_131356
cimequinox Nov 21, 2025
b9470d9
Generated files
cimequinox Nov 21, 2025
d98597e
Move code for ease of review, add comments, improve naming of source …
cimequinox Nov 21, 2025
411de4c
Merge branch 'main' into esql_like_parameters_131356
cimequinox Nov 25, 2025
c486dbc
Merge branch 'main' into esql_like_parameters_131356
cimequinox Nov 26, 2025
a6c96f6
Merge branch 'main' into esql_like_parameters_131356
cimequinox Nov 26, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/changelog/138051.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 138051
summary: Support for parameters in LIKE and RLIKE
area: ES|QL
type: enhancement
issues: []
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,15 @@ ROW message = "foobar"
```


Patterns may be specified with REST query placeholders as well

```esql
FROM employees
| WHERE first_name LIKE ?pattern
| KEEP first_name, last_name
```

```{applies_to}
stack: ga 9.3
```

Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,15 @@ ROW message = "foobar"
```


Patterns may be specified with REST query placeholders as well

```esql
FROM employees
| WHERE first_name RLIKE ?pattern
| KEEP first_name, last_name
```

```{applies_to}
stack: ga 9.3
```

Original file line number Diff line number Diff line change
Expand Up @@ -1858,6 +1858,46 @@ public void testMatchFunctionAcrossMultipleIndicesWithMissingField() throws IOEx
}
}

public void testParamsWithLike() throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a few more tests that cover positional parameters (fields like ?1) and anonymous parameters (fields like ?)? We ran into issues with named parameters with array value (for example, {"pattern": ["a", "b"]}) previously, so it would be great to include some validation tests for that case as well.

Copy link
Contributor Author

@cimequinox cimequinox Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add some use of positional and anonymous parameters to the tests.
I've changed the StatementParserTests, the AnalyzerTests and the RestEsqlTestCase tests to make use of anonymous, positional and named parameters.

It did not occur to me to consider array values but they could be useful.
For example in addition to
field LIKE (?p1, ?p2) with params {"p1": "a*", "p2": "b*"}
we could allow
field LIKE ?parray with params {"parray": ["a*", "b*"]}
but I can imagine this could get complex if we allow both.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It did not occur to me to consider array values but they could be useful. For example in addition to field LIKE (?p1, ?p2) with params {"p1": "a*", "p2": "b*"} we could allow field LIKE ?parray with params {"parray": ["a*", "b*"]} but I can imagine this could get complex if we allow both.

Yes, supporting array parameters could add extra complexity, we got a similar request to support array parameters for inlist predicate previously, and it is not supported yet, so it is not required for this PR, making sure clear error messages are returned is good.

Copy link
Contributor Author

@cimequinox cimequinox Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the PR description suggesting we consider array parameter support in a followup change.

Copy link
Contributor Author

@cimequinox cimequinox Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the base_conversion index out of convenience, I ran a few queries passing arrays as parameters and noticed some curious behavior.

This query works as expected:
{
  "query": "FROM base_conversion | WHERE string LIKE (?, ?) | KEEP string",
  "params": ["f*", "0*"]
}
    string     
---------------
0xff           
ff             
0x32           

but this array query returns no results without generating any error

{
  "query": "FROM base_conversion | WHERE string LIKE ?patterns | KEEP string",
  "params": [{"patterns": ["f*", "0*"]}]
}
    string     
---------------

looking at the plan for each, the first becomes

ExchangeSinkExec[[string{f}#116],false]
\_ProjectExec[[string{f}#116]]
  \_FieldExtractExec[string{f}#116]<[],[]>
    \_EsQueryExec[base_conversion], indexMode[standard], ...
  "esql_single_value" : {
    "field" : "string",
    "next" : {                                  ⎞ LIKE with two patterns
      "expressionQueryBuilder" : {              ⎥                                               
        "field" : "string",                     ⎥                                               
        "expression" : "string LIKE (?, ?)"     ⎠                                               
      }
    },
    "source" : "string LIKE (?, ?)@1:30"
  }
}], tags=[]}]]

but the second becomes

ExchangeSinkExec[[string{f}#120],false]
\_ProjectExec[[string{f}#120]]
  \_FieldExtractExec[string{f}#120]<[],[]>
    \_EsQueryExec[base_conversion], indexMode[standard], ...
  "esql_single_value" : {
    "field" : "string",
    "next" : {
      "term" : {                                ⎞ LIKE pattern ["f*","0*] turned into this
        "string" : {                            ⎥                                               
          "value" : "[[66 2a], [30 2a]]",       ⎥                                               
          "boost" : 0.0                         ⎠                                               
        }
      }
    },
    "source" : "string LIKE ?patterns@1:30"
  }
}], tags=[]}]]

A simple WHERE string LIKE ?pattern with "params": [{"pattern" : "f*"}] becomes

ExchangeSinkExec[[string{f}#128],false]
\_ProjectExec[[string{f}#128]]
  \_FieldExtractExec[string{f}#128]<[],[]>
    \_EsQueryExec[base_conversion], indexMode[standard], ...
  "esql_single_value" : {
    "field" : "string",
    "next" : {
      "wildcard" : {                            ⎞ LIKE pattern "f*" was translated to a wildcard
        "string" : {                            ⎥                                               
          "wildcard" : "f*",                    ⎥                                               
          "boost" : 0.0                         ⎠                                               
        }
      }
    },
    "source" : "string LIKE ?pattern@1:30"
  }
}], tags=[]}]]

I think in the array case it slips by because the "type" of the array is the type of the first element so it gets passed farther than expected.

This PR needs a little more error checking to properly prohibit array valued parameters and generate a useful error.

Copy link
Contributor Author

@cimequinox cimequinox Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added an invalidListParameter function to explicitly check for list parameters and a test to verify they are prohibited in LIKE/RLIKE expressions for now.

With this change the query which previously passed along the stringified list

{
  "query": "FROM base_conversion | WHERE string LIKE ?patterns | KEEP string",
  "params": [{"patterns": ["f*", "0*"]}]
}

now fails as expected.

{
  "error": {
     ...
    "type": "parsing_exception",
    "reason": "line 1:30: Invalid pattern parameter type for LIKE [?patterns]: expected string, found list"
  },
  "status": 400
}

assumeTrue("like parameter support", EsqlCapabilities.Cap.LIKE_PARAMETER_SUPPORT.isEnabled());
bulkLoadTestData(11);
var anonymous_query = requestObjectBuilder().query(
format(null, "from {} | where keyword like ? | keep keyword | sort keyword", testIndexName())
).params("[\"key*0\"]");
Map<String, Object> result = runEsql(anonymous_query);
assertEquals(List.of(List.of("keyword0"), List.of("keyword10")), result.get("values"));
}

public void testParamsWithLikeList() throws IOException {
assumeTrue("like parameter support", EsqlCapabilities.Cap.LIKE_PARAMETER_SUPPORT.isEnabled());
bulkLoadTestData(11);
var positional_query = requestObjectBuilder().query(
format(null, "from {} | where keyword like (?1, ?2) | keep keyword | sort keyword", testIndexName())
).params("[\"key*0\", \"key*1\"]");
Map<String, Object> result = runEsql(positional_query);
assertEquals(List.of(List.of("keyword0"), List.of("keyword1"), List.of("keyword10")), result.get("values"));
}

public void testParamsWithRLike() throws IOException {
assumeTrue("like parameter support", EsqlCapabilities.Cap.LIKE_PARAMETER_SUPPORT.isEnabled());
bulkLoadTestData(11);
var query = requestObjectBuilder().query(
format(null, "from {} | where keyword rlike ?pattern | keep keyword | sort keyword", testIndexName())
).params("[{\"pattern\" : \"key.*0\"}]");
Map<String, Object> result = runEsql(query);
assertEquals(List.of(List.of("keyword0"), List.of("keyword10")), result.get("values"));
}

public void testParamsWithRLikeList() throws IOException {
assumeTrue("like parameter support", EsqlCapabilities.Cap.LIKE_PARAMETER_SUPPORT.isEnabled());
bulkLoadTestData(11);
var query = requestObjectBuilder().query(
format(null, "from {} | where keyword rlike (?p1, ?p2) | keep keyword | sort keyword", testIndexName())
).params("[{\"p1\" : \"key.*0\"}, {\"p2\" : \"key.*1\"}]");
Map<String, Object> result = runEsql(query);
assertEquals(List.of(List.of("keyword0"), List.of("keyword1"), List.of("keyword10")), result.get("values"));
}

@SuppressWarnings("fallthrough")
public void testRandomTimezoneBuckets() throws IOException {
assumeTrue("timezone support for date_trunc is required", EsqlCapabilities.Cap.DATE_TRUNC_TIMEZONE_SUPPORT.isEnabled());
Expand Down
5 changes: 5 additions & 0 deletions x-pack/plugin/esql/src/main/antlr/EsqlBaseParser.g4
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,11 @@ identifierOrParameter
| doubleParameter
;

stringOrParameter
: string
| parameter
;

limitCommand
: LIMIT constant
;
Expand Down
8 changes: 4 additions & 4 deletions x-pack/plugin/esql/src/main/antlr/parser/Expression.g4
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ booleanExpression
;

regexBooleanExpression
: valueExpression (NOT)? LIKE string #likeExpression
| valueExpression (NOT)? RLIKE string #rlikeExpression
| valueExpression (NOT)? LIKE LP string (COMMA string )* RP #likeListExpression
| valueExpression (NOT)? RLIKE LP string (COMMA string )* RP #rlikeListExpression
: valueExpression (NOT)? LIKE stringOrParameter #likeExpression
| valueExpression (NOT)? RLIKE stringOrParameter #rlikeExpression
| valueExpression (NOT)? LIKE LP stringOrParameter (COMMA stringOrParameter )* RP #likeListExpression
| valueExpression (NOT)? RLIKE LP stringOrParameter (COMMA stringOrParameter )* RP #rlikeListExpression
;

matchBooleanExpression
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1688,6 +1688,11 @@ public enum Cap {
*/
TIME_SERIES_WINDOW_V1,

/**
* Support like/rlike parameters https://github.com/elastic/elasticsearch/issues/131356
*/
LIKE_PARAMETER_SUPPORT,

/**
* PromQL support in ESQL, before it is released into tech preview.
* When implementing new functionality or breaking changes,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,30 +31,50 @@ public class RLike extends RegexMatch<RLikePattern> {
public static final NamedWriteableRegistry.Entry ENTRY = new NamedWriteableRegistry.Entry(Expression.class, "RLike", RLike::new);
public static final String NAME = "RLIKE";

@FunctionInfo(returnType = "boolean", description = """
Use `RLIKE` to filter data based on string patterns using
<<regexp-syntax,regular expressions>>. `RLIKE` usually acts on a field placed on
the left-hand side of the operator, but it can also act on a constant (literal)
expression. The right-hand side of the operator represents the pattern.""", detailedDescription = """
Matching special characters (eg. `.`, `*`, `(`...) will require escaping.
The escape character is backslash `\\`. Since also backslash is a special character in string literals,
it will require further escaping.

<<load-esql-example, file=string tag=rlikeEscapingSingleQuotes>>

To reduce the overhead of escaping, we suggest using triple quotes strings `\"\"\"`

<<load-esql-example, file=string tag=rlikeEscapingTripleQuotes>>
```{applies_to}
stack: ga 9.2
serverless: ga
```

Both a single pattern or a list of patterns are supported. If a list of patterns is provided,
the expression will return true if any of the patterns match.

<<load-esql-example, file=where-like tag=rlikeListDocExample>>
""", operator = NAME, examples = @Example(file = "docs", tag = "rlike"))
@FunctionInfo(
returnType = "boolean",
description = """
Use `RLIKE` to filter data based on string patterns using
<<regexp-syntax,regular expressions>>. `RLIKE` usually acts on a field placed on
the left-hand side of the operator, but it can also act on a constant (literal)
expression. The right-hand side of the operator represents the pattern.""",

// we use an inline example here because ?pattern not supported in csv-spec test
detailedDescription = """
Matching special characters (eg. `.`, `*`, `(`...) will require escaping.
The escape character is backslash `\\`. Since also backslash is a special character in string literals,
it will require further escaping.

<<load-esql-example, file=string tag=rlikeEscapingSingleQuotes>>

To reduce the overhead of escaping, we suggest using triple quotes strings `\"\"\"`

<<load-esql-example, file=string tag=rlikeEscapingTripleQuotes>>
```{applies_to}
stack: ga 9.2
serverless: ga
```

Both a single pattern or a list of patterns are supported. If a list of patterns is provided,
the expression will return true if any of the patterns match.

<<load-esql-example, file=where-like tag=rlikeListDocExample>>

Patterns may be specified with REST query placeholders as well

```esql
FROM employees
| WHERE first_name RLIKE ?pattern
| KEEP first_name, last_name
```

```{applies_to}
stack: ga 9.3
```
""",
operator = NAME,
examples = @Example(file = "docs", tag = "rlike")
)
public RLike(
Source source,
@Param(name = "str", type = { "keyword", "text" }, description = "A literal value.") Expression value,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,36 +36,55 @@ public class WildcardLike extends RegexMatch<WildcardPattern> {
);
public static final String NAME = "LIKE";

@FunctionInfo(returnType = "boolean", description = """
Use `LIKE` to filter data based on string patterns using wildcards. `LIKE`
usually acts on a field placed on the left-hand side of the operator, but it can
also act on a constant (literal) expression. The right-hand side of the operator
represents the pattern.

The following wildcard characters are supported:

* `*` matches zero or more characters.
* `?` matches one character.""", detailedDescription = """
Matching the exact characters `*` and `.` will require escaping.
The escape character is backslash `\\`. Since also backslash is a special character in string literals,
it will require further escaping.

<<load-esql-example, file=string tag=likeEscapingSingleQuotes>>

To reduce the overhead of escaping, we suggest using triple quotes strings `\"\"\"`

<<load-esql-example, file=string tag=likeEscapingTripleQuotes>>

```{applies_to}
stack: ga 9.1
serverless: ga
```
Both a single pattern or a list of patterns are supported. If a list of patterns is provided,
the expression will return true if any of the patterns match.

<<load-esql-example, file=where-like tag=likeListDocExample>>

""", operator = NAME, examples = @Example(file = "docs", tag = "like"))
@FunctionInfo(
returnType = "boolean",
description = """
Use `LIKE` to filter data based on string patterns using wildcards. `LIKE`
usually acts on a field placed on the left-hand side of the operator, but it can
also act on a constant (literal) expression. The right-hand side of the operator
represents the pattern.

The following wildcard characters are supported:

* `*` matches zero or more characters.
* `?` matches one character.""",

// we use an inline example here because ?pattern not supported in csv-spec test
detailedDescription = """
Matching the exact characters `*` and `.` will require escaping.
The escape character is backslash `\\`. Since also backslash is a special character in string literals,
it will require further escaping.

<<load-esql-example, file=string tag=likeEscapingSingleQuotes>>

To reduce the overhead of escaping, we suggest using triple quotes strings `\"\"\"`

<<load-esql-example, file=string tag=likeEscapingTripleQuotes>>

```{applies_to}
stack: ga 9.1
serverless: ga
```
Both a single pattern or a list of patterns are supported. If a list of patterns is provided,
the expression will return true if any of the patterns match.

<<load-esql-example, file=where-like tag=likeListDocExample>>

Patterns may be specified with REST query placeholders as well

```esql
FROM employees
| WHERE first_name LIKE ?pattern
| KEEP first_name, last_name
```

```{applies_to}
stack: ga 9.3
```
""",
operator = NAME,
examples = @Example(file = "docs", tag = "like")
)
public WildcardLike(
Source source,
@Param(name = "str", type = { "keyword", "text" }, description = "A literal expression.") Expression left,
Expand Down

Large diffs are not rendered by default.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Large diffs are not rendered by default.

Loading
Loading