Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filtered aliases in templates do not inherit mappings from aliased index #8473

Closed
loren opened this issue Nov 13, 2014 · 3 comments
Closed
Assignees

Comments

@loren
Copy link

loren commented Nov 13, 2014

Related to #6110 and #8431. This behavior is new since 1.4.0.

#reset logstash templates
DELETE /_template/logstash
DELETE /_template/filtered_logstash

# ensure index does not exist
DELETE logstash-2014.11.11

#create template for logstash indexes
PUT /_template/logstash
{
  "template": "logstash-*",
  "mappings": {
    "_default_": {
      "dynamic_templates": [
        {
          "string_fields": {
            "mapping": {
              "index": "not_analyzed",
              "type": "string"
            },
            "match_mapping_type": "string",
            "match": "*"
          }
        }
      ],
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "clientip": {
          "index": "not_analyzed",
          "type": "string"
        }
      },
      "_all": {
        "enabled": true
      }
    }
  }
}

PUT _template/filtered_logstash
{
  "template": "logstash-*",
  "order": 1,
  "aliases": {
    "filtered-{index}": {
      "filter": {
        "bool": {
          "must_not": [
            {
              "terms": {
                "clientip": [
                  "1.2.3.4",
                  "2.3.4.5"
                ]
              }
            }
          ]
        }
      }
    }
  }
}

# (FAILS) insert doc to dynamically create index
POST /logstash-2014.11.11/event/1
{
  "@timestamp": "2014-11-11T00:02:14.000Z",
  "clientip": "3.4.5.6"
}

I would expect this to work, but instead I get Strict field resolution and no field mapping can be found for the field with name [clientip] when I try to POST that event.

@clintongormley
Copy link

@loren thanks for opening this. The recreation can be simplified to a single template. It appears that the _default_ mapping is only consulted after the aliases have been created:

DELETE _all 
DELETE /_template/*

PUT /_template/logstash
{
  "template": "logstash-*",
  "mappings": {
    "_default_": {
      "properties": {
        "clientip": {
          "index": "not_analyzed",
          "type": "string"
        }
      }
    }
  },
  "aliases": {
    "filtered-{index}": {
      "filter": {
        "bool": {
          "must_not": [
            {
              "terms": {
                "clientip": [
                  "1.2.3.4",
                  "2.3.4.5"
                ]
              }
            }
          ]
        }
      }
    }
  }
}

Throws Strict field resolution and no field mapping can be found for the field with name [clientip]:

POST /logstash-2014.11.11/event/1
{
  "clientip": "3.4.5.6"
}

However, if you change the _default_ mapping in the template to event, then this works just fine.

@martijnvg could you have a look please?

martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Nov 24, 2014
…should be picked up when an index alias filter is parsed if a new index is introduced when a document is indexed into an index that doesn't exist yet.

Closes elastic#8473
martijnvg added a commit that referenced this issue Nov 24, 2014
…should be picked up when an index alias filter is parsed if a new index is introduced when a document is indexed into an index that doesn't exist yet.

Closes #8473
@shivangshah
Copy link

I apologize in advance for opening this conversation up after months of it being resolved, but I still see this problem with version 1.4.4. I have detailed the usecase in question and the corresponding files here: http://stackoverflow.com/questions/28977610/elastic-search-filtered-aliases-with-unmapped-fields
All help is appreciated and I apologize in advance if it is something silly that I missed !

@eugenp
Copy link

eugenp commented Mar 12, 2015

Quick note - I am seeing the same problem after upgrading to 1.4.4.

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015
…should be picked up when an index alias filter is parsed if a new index is introduced when a document is indexed into an index that doesn't exist yet.

Closes elastic#8473
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants