Give bulk requests the ability to give hints to dynamic mappings

[jpountz]

This feature consists of enabling dynamic templates to register arbitrary match_mapping_types, e.g.

PUT myindex
{
  "mappings": {
    "dynamic_templates": [{
      "time_histograms": {
        "match_mapping_type": "time_histogram", // defines a `time_histogram` type
        "mapping": {
          "type": "histogram",
          "meta": {
            "unit": "s"
          }
        }
      }
    }]
  }
}

And then allow bulk requests to tell Elasticseach what dynamic template to apply for a given field

POST myindex/_bulk
{ "index": { "match_mapping_types": { "response_times": "time_histogram" } } }
{ "@timestamp": "2020-08-12", "response_times": { "values": [1, 10], "counts": [5, 1] }}

This helps when ingesting data whose schema can't be known up-front, such as when collecting metrics from Promotheus or Statsd.

This syntax is an example, it's not set in stone, but the following points were considered:

• Type hints are provided as part of the bulk request. Requiring a separate request to another API is not an option as we would lose atomicity (e.g. think of an alias being swapped) and ability to integrate with ingest processors.
• The bulk request sends a match_mapping_type, not actual mappings. Sending actual mappings would be problematic as data shippers typically don't have enough privilege to do mapping updates.
• Each index request within the bulk can set its own hints. Configuring this at the bulk level could be nice at some point, but we plan to use this feature via Ingest, at least initially, which works at the index request level.

[elasticmachine]

Pinging @elastic/es-search (:Search/Mapping)

[ruflin]

If we have this feature, we could standardise on some match_mapping_types names and use the in the metrics-*-* base template. This would mean most metric ingestion with type hints just work out of the box.

[cyrille-leclerc]

Thank you very much @dnhatn . This histogram feature will greatly help observability!