Alias filters cached/type inferenced incorrectly

[dbertram]

A while ago we started playing with alias filters, but initially ran into a confusing bug.

Our environment creates our index along with its type mappings and aliases all at once, then proceeds to index documents into it.

The problem lies with using a numeric filter in an alias on a field that does not have an explicit type via its mapping. After indexing documents that match the alias' filter, no results are returned. This appears to be due to the alias being cached.

The following is a minimal reproduction of the issue:

curl -XPUT 'http://localhost:9200/testindex/'

echo create alias that filters by numericField:1
curl -XPOST 'http://localhost:9200/_aliases' -d '{
  "actions": [
    {
      "add": {
        "index": "testindex",
        "alias": "testalias",
        "filter": { "term": { "numericField": 1 } }
      }
    }
  ]
}'

echo index a case that should match our filter
curl -XPUT 'http://localhost:9200/testindex/testtype/1' -d '{
  "name": "bob",
  "numericField": 1,
  "message": "testing numeric alias filters"
}'

echo confirm we can find the document via term filter
curl -XPOST 'http://localhost:9200/testindex/_search' -d '{
  "filter": {
    "term": { "numericField": 1 }
  }
}'

echo document not matched when filtering via the alias
curl -XPOST 'http://localhost:9200/testalias/_search' -d '{
  "query": {
    "match_all": {}
  }
}'

Inspecting the alias via the _aliases endpoint doesn't reveal any information that would explain the lack of matching documents.

If you explicitly type the filtered field in the mapping before creating the alias, the alias filter works as expected.

curl -XPOST 'http://localhost:9200/testindex/' -d '{
  "mappings": {
    "testtype": {
      "properties": {
        "numericField": { "type": "long" }
      }
    }
  }
}'

Given that this was relatively tricky to figure out (nothing is mentioned about type inferencing in relation to alias filters in the docs), this struck us as being a bug in how alias filters are being stored/cached.

[s1monw]

hey,

the reason here is not caching I am afraid. This would be simpler to fix :) The problem here is that we parse the filter from the alias and then pre-build it with the "current" knowledge about the field. Since there is no mapping we can't tell that we need to build a numeric filter which is encoded differently in the index. From the top of my head this is not easy to fix really but I will think about it. I guess the easiest fix for you are dynamic templates that add a mapping for those fields on index creation you can maybe prefix them with numierc_ or something like that?

[dbertram]

I had a feeling it might not be as straightforward as a caching issue.

For us, this is easy to work around as the field we want to filter on is known at index creation time (user account number), so we can just specify it explicitly in our type mapping. I just wanted to report it as it was something that tripped us up and I'm sure other people might run into it.

Couple of ideas/questions (not knowing the guts of ES):

1. Why aren't the alias filters type inferenced the same way the document fields are when being indexed? The example is specifying a numeric term filter (i.e., "numericField": 1, not "numericField": "1"). Shouldn't that be sufficient for ES to know to build a numeric filter?
2. Could the type mismatch between the alias filter and the underlying field in the index be detected at search time (and repaired/reported via an error)?
3. When a document is indexed that updates the type mapping, could affected alias filters be invalidated/rebuilt as appropriate?

[tmkujala]

I'm also seeing this issue on 0.90.3. While adding an explicit type mapping for the filtered field works, it appears that a default mapping does not. I'm leaning toward adding a specific type mapping for the field in addition to the default type mapping for dynamic types.

Here's a brief recreation:

# create a new dated index
curl -XPUT http://localhost:9200/test_index_20140129

# add a default mapping
curl -XPUT http://localhost:9200/test_index_20140129/_default_/_mapping -d '{
  "_default_" : {
    "properties" : {
      "@timestamp" : {
        "type" : "date",
        "format" : "dateOptionalTime"
      }
    }
  }
}'

# create an alias to the index with a time range filter
curl -XPUT http://localhost:9200/test_index_20140129/_alias/alias_test_index_20140129 -d '{
  "filter" : {
    "range" : {
      "@timestamp" : {
        "lt" : "2014-01-30T00:00:00-0500",
        "gte" : "2014-01-29T00:00:00-0500"
      }
    }
  }
}'

# index a document
curl -XPOST http://localhost:23392/alias_test_index_20140129/test -d '{
  "@timestamp" : "2014-01-29T12:00:00-0500",
  "@message" : "this is a test"
}'

# try to search for the document via the alias
curl -XGET http://localhost:9200/alias_test_index_20140129/_search?pretty

[clintongormley]

Closing in favour of #6664