Automatically flatten objects when subobjects: false

[felixbarny]

Currently, when an object is mapped with subobjects: false, trying to ingest a document that has nested objects leads to a mapping exception.

In a couple of occasions, we've been talking about that it would be convenient if Elasticsearch could automatically flatten the fields instead of the client having to know about the subobjects setting. This would make subobjects: false more of a drop-in replacement that allows for a much easier adoption, without changes in the client.

Note that the _source should ideally stay untouched, just the stored fields should be flattened automatically.

Some related issues

• Support metrics with dots in their names apm#347
• Better default mappings for logs #88777
• Add support for subobjects: false package-spec#349

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[javanna]

Note that the _source should ideally stay untouched, just the stored fields should be flattened automatically.

Correct, the idea is that the _source would stay the same, only the corresponding mapping representation would change, so that intermediate objects would not be dynamically mapped as such, but rather only the leaf fields would be mapped with their full path as their name instead.

Note that this is only about incoming documents. Incoming mappings can't hold objects whenever subobjects is set to false, and that will stay as-is.

[felixbarny]

Not sure if I understand this correctly:

Incoming mappings can't hold objects whenever subobjects is set to false, and that will stay as-is.

Did you mean that trying to explicitly map a property as type: object doesn't work if subobjects is false? That's what I would have expected. Or is there something more than that?

[eyalkoren]

Are there cases that will require special handling?
For example, given this mapping:

PUT my-index-000001
{
  "mappings": {
    "properties": {
      "location": {
        "type": "geo_point"
      }
    }
  }
}

one can ingest a location as follows:

PUT my-index-000001/_doc/1
{
  "location": { 
    "lat": 41.12,
    "lon": -71.34
  }
}

Should there be special handling so that such won't be automatically flattened?

[javanna]

special handling so that such won't be automatically flattened?

The special handling would be done by Elasticsearch based on mappings definition. This is one of the complex aspects that made us consider this a feature of Elasticsearch as opposed to implemented on the client side: you can't blindly flatten everything on the sender side.