Defining default_index and default_search analyzers sets type-level index and search analyzers

[rayward]

The migration plugin is complaining that I have type-level analyzers configured for all my types:

Type-level analyzer settings
analyzer, search_analyzer and index_analyzer settings have been removed and will use the index defaults instead, in type: foo.

I've pinned it down to the default_index and default_search definitions on the index analysis settings.

curl -XPUT localhost:9200/test -d '
{
  "index": {
    "analysis": {
      "analyzer": {
        "default_search": {
          "filter": [
            "standard",
            "lowercase",
            "stop",
            "asciifolding"
          ],
          "tokenizer": "standard",
          "type": "custom"
        },
        "default_index": {
          "filter": [
            "standard",
            "lowercase",
            "stop",
            "asciifolding"
          ],
          "tokenizer": "standard",
          "type": "custom"
        }
      }
    }
  }
}
'

curl -XPUT localhost:9200/test/foo/_mapping -d '{"foo": {"properties":{}}}'

curl -XGET localhost:9200/test/foo/_mapping?pretty

{
  "test" : {
    "mappings" : {
      "foo" : {
        "index_analyzer" : "default_index",
        "search_analyzer" : "default_search",
        "properties" : { }
      }
    }
  }
}

Is the how the intended behaviour of default_index and default_search (to set type-level analyzers)?

Does this mean in 2.0 I have to explicitly set the analyzers for all string fields now rather than being able to use a default that I define?

Perhaps the docs for default analyzers also needs updating then?
https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-analyzers.html#default-analyzers

[rjernst]

@rayward You can certainly still define a default analyzer. Previously there were two different levels of default analyzer you could set. The first was on the entire index. The second was on each type. The latter was removed with 2.0 (see the reasoning in #8874), but the default analyzer can still be set on the entire index.

If you remove the _index suffix from your analyzer name (that is going away with #11861), and remove the index_analyzer and search_analyzer settings from your type, the migration tool should be happy. default and default_search are special names for analyzer. If you don't specify an analyzer for a field, it will use these.

[rayward]

@rjernst thanks for clarifying, I think it's just a bit confusing because I never explicitly specified the type-level analyzers on the mapping, ES picked it up from from the index level settings.

I'm not even sure how to even remove them actually (perhaps because they're not actually set on the

curl -XPUT localhost:9200/test/foo/_mapping -d '{"foo": {"properties":{}, "search_analyzer": null}}'
{"error":"NullPointerException[null]","status":500}

curl -XPUT localhost:9200/test/foo/_mapping -d '{"foo": {"properties":{}, "search_analyzer": ""}}'
{"error":"MapperParsingException[Analyzer [] not found for search_analyzer setting on root type [foo]]","status":400}

Nor will it even let me change it:

curl -XPUT localhost:9200/test/foo/_mapping -d '{"foo": {"properties":{}, "search_analyzer": "standard"}}'

The mapping still looks the same:

curl -XGET localhost:9200/test/foo/_mapping?pretty
{
  "test" : {
    "mappings" : {
      "foo" : {
        "index_analyzer" : "default_index",
        "search_analyzer" : "default_search",
        "properties" : { }
      }
    }
  }
}

Am I doing something fundamentally wrong?

[rjernst]

It sounds like the get mappings on 1.x always output the type level if it was "set", even if by specifying an index level default. I think this should actually work when upgrading (it will just be ignored). It looks like this is a test index. Would you mind trying the upgrade and reporting back? @clintongormley can speak to how we might improve the migration plugin to detect this situation (and not error in that case).

[rjernst]

I'll reopen for now, until we determine how to improve the migration plugin and/or if there is work needed to allow the upgrade error free in this case.

[clintongormley]

@rayward thanks for pointing this out. I'll move the issue to the migration repo and address this there

[clintongormley]

This issue was moved to elastic/elasticsearch-migration#23