Unable to sort if a field doesn't exist

[prkadalbcisco]

Elasticsearch Version

8.19.7

Installed Plugins

No response

Java Version

bundled

OS Version

Mac x86_64

Problem Description

When searching multiple indices and a field only exists in ONE of the indices, elasticsearch should allow sorting by that field, regardless of the "unmapped" type provided.

This was the behaviour in 7.17, and a fix was made earlier, but the fix appears to be incomplete.

#102779

Steps to Reproduce

Bash script to reproduce the problem -
test1.sh

Output from 7.17

=== Get Elasticsearch Version ===
"7.17.0"

=== Clean up: Delete test indices if they exist ===
Deleted index: testindex1
Deleted index: testindex2

=== Step 1: Create two indices ===
{"acknowledged":true,"shards_acknowledged":true,"index":"testindex1"}
{"acknowledged":true,"shards_acknowledged":true,"index":"testindex2"}
=== Step 2: Index a document with a double field into testindex1 ===
{"_index":"testindex1","_type":"_doc","_id":"1","_version":1,"result":"created","forced_refresh":true,"_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
=== Step 3: Search both indices, sorting by double_field, unmapped_type=keyword ===
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [
      {
        "_index": "testindex1",
        "_type": "_doc",
        "_id": "1",
        "_score": null,
        "_source": {
          "double_field": 3.14
        },
        "sort": [
          3.14
        ]
      }
    ]
  }
}

=== Step 4: Index a document without double_field into testindex2 ===
{"_index":"testindex2","_type":"_doc","_id":"1","_version":1,"result":"created","forced_refresh":true,"_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
=== Step 5: Rerun the same search ===
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [
      {
        "_index": "testindex2",
        "_type": "_doc",
        "_id": "1",
        "_score": null,
        "_source": {
          "other_field": "foo"
        },
        "sort": [
          null
        ]
      },
      {
        "_index": "testindex1",
        "_type": "_doc",
        "_id": "1",
        "_score": null,
        "_source": {
          "double_field": 3.14
        },
        "sort": [
          3.14
        ]
      }
    ]
  }
}

Output from 8.19

=== Get Elasticsearch Version ===
"8.19.7"

=== Clean up: Delete test indices if they exist ===
Index testindex1 did not exist or could not be deleted (HTTP 404)
Index testindex2 did not exist or could not be deleted (HTTP 404)

=== Step 1: Create two indices ===
{"acknowledged":true,"shards_acknowledged":true,"index":"testindex1"}
{"acknowledged":true,"shards_acknowledged":true,"index":"testindex2"}
=== Step 2: Index a document with a double field into testindex1 ===
{"_index":"testindex1","_id":"1","_version":1,"result":"created","forced_refresh":true,"_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
=== Step 3: Search both indices, sorting by double_field, unmapped_type=keyword ===
{
  "took": 72,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [
      {
        "_index": "testindex1",
        "_id": "1",
        "_score": null,
        "_source": {
          "double_field": 3.14
        },
        "sort": [
          3.14
        ]
      }
    ]
  }
}

=== Step 4: Index a document without double_field into testindex2 ===
{"_index":"testindex2","_id":"1","_version":1,"result":"created","forced_refresh":true,"_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
=== Step 5: Rerun the same search ===
{
  "error": {
    "root_cause": [],
    "type": "search_phase_execution_exception",
    "reason": "",
    "phase": "fetch",
    "grouped": true,
    "failed_shards": [],
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "Can't sort on field [__anonymous_]; the field has incompatible sort types: [FLOAT] and [STRING] across shards!"
    }
  },
  "status": 400
}

=== Test complete ===

Output from 8.10 for reference (where the previous incomplete fix isn't available)

=== Get Elasticsearch Version ===
"8.10.2"

=== Clean up: Delete test indices if they exist ===
Deleted index: testindex1
Deleted index: testindex2

=== Step 1: Create two indices ===
{"acknowledged":true,"shards_acknowledged":true,"index":"testindex1"}
{"acknowledged":true,"shards_acknowledged":true,"index":"testindex2"}
=== Step 2: Index a document with a double field into testindex1 ===
{"_index":"testindex1","_id":"1","_version":1,"result":"created","forced_refresh":true,"_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
=== Step 3: Search both indices, sorting by double_field, unmapped_type=keyword ===
{
  "error": {
    "root_cause": [],
    "type": "search_phase_execution_exception",
    "reason": "",
    "phase": "fetch",
    "grouped": true,
    "failed_shards": [],
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "Can't sort on field [__anonymous_]; the field has incompatible sort types: [FLOAT] and [STRING] across shards!"
    }
  },
  "status": 400
}

=== Step 4: Index a document without double_field into testindex2 ===
{"_index":"testindex2","_id":"1","_version":1,"result":"created","forced_refresh":true,"_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
=== Step 5: Rerun the same search ===
{
  "error": {
    "root_cause": [],
    "type": "search_phase_execution_exception",
    "reason": "",
    "phase": "fetch",
    "grouped": true,
    "failed_shards": [],
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "Can't sort on field [__anonymous_]; the field has incompatible sort types: [FLOAT] and [STRING] across shards!"
    }
  },
  "status": 400
}

=== Test complete ===

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-search-foundations (Team:Search Foundations)

[spinscale]

Hey, thanks for the detailed report and the ability to reproduce. Much appreciated!

I have one question on your example. You configured the unmapped_type to be a keyword. However the other index has a float/double for the double_field - those are two incompatible data types, and thus the exception.

If you change your unmapped_type to float this seems to work as expected to me?

For further reference, if others want to test, this is a quick integration test

    /* this one fails */
    public void testSortError() {
        assertAcked(
            prepareCreate("test1"),
            prepareCreate("test2")
        );

        client().prepareBulk().setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE)
            .add(prepareIndex("test1").setId("1").setSource("field", 1.0))
            .add(prepareIndex("test2").setId("2").setSource("other", 1.0))
            .get();

        assertNoFailures(
            prepareSearch("test1", "test2").addSort(new FieldSortBuilder("field").unmappedType("keyword"))
        );
    }

    /* this one works */
    public void testSortWorks() {
        assertAcked(
            prepareCreate("test1"),
            prepareCreate("test2")
        );

        client().prepareBulk().setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE)
            .add(prepareIndex("test1").setId("1").setSource("field", 1.0))
            .add(prepareIndex("test2").setId("2").setSource("other", 1.0))
            .get();

        assertNoFailures(
            prepareSearch("test1", "test2").addSort(new FieldSortBuilder("field").unmappedType("float"))
        );
    }

[prkadalbcisco]

You configured the unmapped_type to be a keyword. However the other index has a float/double for the double_field - those are two incompatible data types, and thus the exception.

Yes, this is exactly the situation in which 7.x does not throw an exception, and this was the same problem which the other issue I referenced had dealt with.
#102723 (comment)

Basically, if there are no docs in the other index with the sort field at all, we don't run into any issues. This is the changed behavior, we are eagerly checking even if there are no docs with the field that mismatch the types.

There is no doc in the other index here as well having a value for that field.