6.x index template data type fallback

[ebeahan]

Summary

Adds logic when generating the 6.x index template artifact to fallback on a compatible type if one exists. Soon, ECS fields will begin using data types introduced in Elasticsearch 7.x, and this change will ensure the ES 6 template will only use types available in 6.x.

Background & Considerations

The first considered approach was reusing the --oss fallback mechanism to generate a second, 6.x-compatible copy of the fields object. However, the OSS-fallback takes place much earlier in the chain and would have required a copy of the fields objects to go through the identical chain of steps as the original fields object.

Next, I looked at incorporating a function to pass along to visitor.visit_fields from generators.es_template, but es_template uses the flat representation incompatible with visitor.visit_fields without refactoring.

Approach

This approach is a variation of the earlier-used visitor function but adopted for the field mappings generated in the ES template genator.

Two details:

• TYPE_FALLBACKS is imported from schema.oss to avoid defining the available fallbacks in two places.
• Another pass through schema.cleaner.field_or_multi_field_datatype_defaults is necessary to restore any missing defaults after updating the type (e.g., wildcard => keyword will need ignore_above added).

I want to revisit this approach in the future. New types will be introduced in later major versions that don't exist in 7.x, and types with no fallback will also need handling.

[ebeahan]

@elasticmachine test this please

[webmat]

I love the fact that this is self-contained in es_template.py!

I have one nit-pick below, but feel free to ignore. Your call :-)

LGTM

[webmat]

Ah yes, the changelog! I was about to point this out 😉