Improve top-level docs on mapping field types.

[jtibshirani]

As we’ve added more field types, the list of mapping types has grown quite long. For users, the list of options can be a bit overwhelming and hard to navigate.

To help address this, we could reorganize the reference on field types:

• Instead of the sections 'core', 'complex', and 'specialized', they could be grouped more by use case. For example there would be a 'full-text search' section that contains text, completion, and search_as_you_type, etc.
• We could add a 'how to' page on choosing the right mappings. The assumption is that the user has already played around with Elasticsearch with all data mapped dynamically, and now wants to design the right schema.

Other suggestions are very welcome!

[elasticmachine]

Pinging @elastic/es-search (:Search/Mapping)

[elasticmachine]

Pinging @elastic/es-docs (>docs)

[jtibshirani]

A couple other ideas:

• Now that we added the notion of a 'field type family' in Collapse field types into "families" of field types in _field_caps. #53175, it would be good to surface these families in the docs. For now, keyword, constant_keyword, and wildcard could be grouped together. We can explain that these field types are guaranteed to support the same functionality, but give different trade-offs in terms of performance.
• In the overview page, we could indicate which field types are OSS vs. licensed (?)

[markharwood]

it would be good to surface these families in the docs

I added a short comment in the field caps docs but I imagine it could be expanded on.

[jrodewig]

I've been working on this in #61117 and stumbled across this reference in the Beats dev guide:

metricbeat/module/{module}/{metricset}/_meta/fields.yml: Contains all fields definitions retrieved by the metricset. As field types, each field must have a core data type supported by elasticsearch. Here’s a very basic example that shows one group from the MySQL status metricset:

https://www.elastic.co/guide/en/beats/devguide/current/metricset-details.html#_fields_yml

I don't think core types has/had any formal or technical meaning. It seems like it just started as a way to group "primitive" data types. However, I've also heard there are some field types that aren't supported by Metricbeat, such as histogram. Removing the core listing could break those docs and make it difficult for users to know what types are supported.

@jtibshirani @dedemorton What are your thoughts? I like the idea of better organizing these types around family/use case. However, that may mean maintaining a list of unsupported field types in the Metricbeat docs.

[jtibshirani]

I don't think core types has/had any formal or technical meaning... However, that may mean maintaining a list of unsupported field types in the Metricbeat docs.

Depending on the new grouping, it could make sense to retain a section for 'core' data types. But I agree that 'core' is not meant to be an official concept, and we regularly add (and may remove) field types from this section. It feels more robust for Metricbeat to not rely on this section and instead explain what field types are supported/ unsupported.

[jrodewig]

It feels more robust for Metricbeat to not rely on this section and instead explain what field types are supported/ unsupported.

Opened elastic/beats#20671 to capture this.

[jtibshirani]

I'm going to close now that @jrodewig merged #53175 to reorganize the field types and define type families. We also opened #61441 to track combining all the types in the keyword family into the same page.

It'd be great to provide more guidance on choosing the right mappings (through a 'how to' page, visual guide, or even a 'mappings suggestion API'?), but we can plan to address that separately.