Skip to content

ESQL: begins_with and ends_with on _index are not working #130851

Open
@julian-elastic

Description

@julian-elastic

Elasticsearch Version

9.2

Installed Plugins

No response

Java Version

bundled

OS Version

mac

Problem Description

Begins_with and ends_with on _index are not working and always return false.
There are 2 problems:

  1. We create a WildcardQuery with forceStringMatch == false in EndsWith.asQuery().
    We should pass the true flag or use pushdownPredicates.flags().stringLikeOnIndex()
  2. In EndsWith.asQuery() we do
    var wildcardQuery = "*" + QueryParser.escape(BytesRefs.toString(suffix.fold(FoldContext.small())));
    That is the wrong escape, as it will do escape for RegularExpressions. We are creating a wildcard, so we should use wildcard escaping as in WildcardFieldMapper.escapeWildcardSyntax()

Similar for BeginsWith.

Steps to Reproduce

Add the following test to MultiClustersIT.java. We expect to get all remote docs, because the remote index really ends with "remote-index", but we get no data back.

public void testEndsWithIndex() throws Exception {
        boolean includeCCSMetadata = includeCCSMetadata();
        Map<String, Object> result = run("""
            FROM test-local-index,*:test-remote-index METADATA _index
            | WHERE ENDS_WITH(_index, "remote-index")
            | STATS c = COUNT(*) BY _index
            | SORT _index ASC
            """, includeCCSMetadata);
        var columns = List.of(Map.of("name", "c", "type", "long"), Map.of("name", "_index", "type", "keyword"));
        var values = List.of(List.of(remoteDocs.size(), REMOTE_CLUSTER_NAME + ":" + remoteIndex));
        assertResultMap(includeCCSMetadata, result, columns, values, false);
    }

Logs (if relevant)

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions