Shard failures when searching multiple indices with a type filter and sort field

[ddecola]

When executing a search with both a type filter and a sort across multiple indices any shards that do not contain the target type mapping fail. While these failures can be masked by using ignore_unmapped, it seems as if the type filter should make the flag unnecessary.

For example, let there be an index aaa with a type mapping Y and another index bbb without mapping Y. A search across all indices for all records with type Y (i.e. GET /_all/Y) and sorted by a field F in Y will fail on all shards containing data for index bbb. The failed shards all throw SearchParseExceptions because there was no mapping Y found for the field F (the sort field).

This failure arises specifically because the sort field is resolved (for type information, etc) before the type filter (or any other filter) is applied. After the filters are applied there are no matching records and so no sort is needed. There does not appear to be any downside to deferring this field resolution until after the number of matched records is known to be greater than zero.

Simple test case for scenario above:

curl -XPUT 'localhost:9200/aaa' -d '{
    "settings" : {
        "number_of_shards" : 5
    },
    "mappings" : {
        "Y" : {
            "properties" : {
                "F" : { "type" : "integer" }
            }
        }
    }
}'

curl -XPUT 'localhost:9200/bbb' -d '{
    "settings" : {
        "number_of_shards" : 5
    }
}'

curl -XPUT 'localhost:9200/aaa/Y/1' -d '{
    "F" : 1
}'

curl -XPOST 'localhost:9200/_all/Y/_search' -d '{
     "sort" : [
        { "F" : {"order" : "asc"} }
    ]
}'

[clintongormley]

We should support ignore_missing for cases where the field doesn't exist in the mapping as well.

[clintongormley]

Also see #2801