Support include_defaults in the GET-mapping API

[clintongormley]

The get-field-mapping API only works for core field types, not for fields of type object or nested:

curl -XPUT "http://localhost:9200/test" -d'
{
  "mappings": {
    "test": {
      "properties": {
        "foo": {
          "type": "object",
          "properties": {
            "bar": {
              "type": "string"
            }
          }
        }
      }
    }
  }
}'

Retrieving the mapping for core fields works:

curl -XGET "http://localhost:9200/test/_mapping/test/field/foo.bar"

Result:

{
   "test": {
      "mappings": {
         "test": {
            "foo.bar": {
               "full_name": "foo.bar",
               "mapping": {
                  "bar": {
                     "type": "string"
                  }
               }
            }
         }
      }
   }
}

But not for fields of type object or nested:

curl -XGET "http://localhost:9200/test/_mapping/test/field/foo"

Result:

{}

[bleskes]

This tricky as the get field mappings api also supports wild cards ( #4367 ) which people use to get a list of all searchable fields. This is handy for suggestions of field names (i.e., a kibana panel facet configuration).

Also - if you specify curl -XGET "http://localhost:9200/index/type/_mapping/field/*" it's ambiguous what you'd get - object fields only ? object fields and leaf fields unpacked?

we could potentially add flags for this but I'm not sure it's worth it. Perhaps a documentation issue or find better url structure (using a clearer word than fields)?

[clintongormley]

The particular use case I had was wanting to see if include_in_parent|root were being defaulted to false. There is no way of seeing this currently.

For wildcards, I'd say you include the object plus all of its properties.

This tricky as the get field mappings api also supports wild cards ( #4367 ) which people use to get a list of all searchable fields. This is handy for suggestions of field names (i.e., a kibana panel facet configuration).

Perhaps just a query string flag to distinguish between searchable fields and all fields, eg:

GET /index/_mapping/type/field/*?searchable

[Mpdreamz]

I have to agree with @clintongormley it would be great if it did support object|nested this endpoint is great to zoom in whether a field was indeed properly mapped without the surrounding noise the regular _mapping endpoint gives you.

When I first read the documentation I did not equate field with leaf fields.

GET /index/_mapping/type/field/*?leafs

or since the default is already to only show leafs:

GET /index/_mapping/type/field/*?include_object

Since wildcards already flatten the result it would be great if the mapping for objects excludes its own properties.

[clintongormley]

I think the main thing that is missing here is support for include_defaults on the main GET-mapping API. While getting a single object mapping to check that it is mapped correctly would be nice, it's a one-off debugging request, and is not as important as the current function of the GET-field-mapping: returning all searchable fields as a flat list.

So I'd be happy with just adding support for include_defaults to GET-mapping

[Mpdreamz]

If that toggles including object and nested fields as well +1

[clintongormley]

@Mpdreamz the change i'm suggesting is to the main get-mapping API, not the get-field-mapping API. So my request doesn't allow you to retrieve a single object field, you still have to retrieve the old mapping. However, you can get back the mapping with the defaults included.

[bleskes]

I'm +1 on adding an include default + filtering options (including indexed, which is i think is better than searchable because it's the name of the setting, nested or what ever) . Once we have that, we can depreciate the get field mappings API imho.

Caveat - The get field mapping api was added and implemented in the current form (i.e., requiring at least one assigned shard) because the defaults are currently baked in the the field mapper in memory objects. To so on the master and based on the serialized representation of the _mapping, we need to have a better grip on the defaults (or if we just choose to instantiate these objects with every request).

[csezheng]

+1 to add the include_defaults option to get mapping API, and also makes it work for the object type field.

[csezheng]

If this is not gonna be fixed soon, is there any alternative way to get the default values for the object/nested type fields? Thanks.

[jpountz]

We just discussed it in Fixit Friday and are +1 on @bleskes 's plan. Making it an adoptme as well as a high-hanging fruit because of the described caveats...

[clintongormley]

Rediscussed in FixItFriday - would be good to:

• remove the field mapping API
• add support for include_defaults to the mapping API
• users can use response filtering for a particular field
• perhaps we can support flat_paths to return fields like foo.bar.baz instead of using JSON objects

@rjernst is the following still true after the mapping refactoring you did?

Caveat - The get field mapping api was added and implemented in the current form (i.e., requiring at least one assigned shard) because the defaults are currently baked in the the field mapper in memory objects. To so on the master and based on the serialized representation of the _mapping, we need to have a better grip on the defaults (or if we just choose to instantiate these objects with every request).

[jpountz]

cc @elastic/es-search-aggs

[mayya-sharipova]

The issue of still relevant. cc @elastic/es-search-aggs

[javanna]

It's been a long time since this issue was discussed last. The get field mappings API has effectively been replaced by the field caps API. We have not removed the API due to the breaking nature of such a change but we would expect users to rely on field_caps to inspect the fields and their capabilities. In case mappings need to be inspected instead, get mappings is the way to go. As for adding support for include_defaults to get mappings, I doubt that it would be useful today given that most users should rely on field_caps, hence I am going to close this issue. Feel free to comment or reopen if you think otherwise.