dynamic_date_formats ignore 'basicDateTimeNoMillis' date format

[PolinaMash]

In my app I strongly use dynamic nature of ElasticSearch, but for dates it looks like not working properly. In my app all dates are stored in yyyyMMdd’T'HHmmssZ format, so I simply want dynamic date detection to use basicDateTimeNoMillis format. But neither dynamic_date_formats nor dynamic template for dates doesn't work. Dates are simply ignored and detected as strings. Here is part of mapping for default type:

       "_default_" : {
            "dynamic_date_formats":"basicDateTimeNoMillis",     
            "properties" : {
                "uri" : {
                    "type" : "string"
                },
                "sn" : {
                    "type" : "string"
                },
                "acl" : {
                    "type" : "object"
                }               
            },
            "dynamic_templates": [
                { "date": {
                    "path_match": "*",
                    "match_mapping_type": "date",
                    "mapping": {
                    "type": "date",
                    "format": "basicDateTimeNoMillis"
                }
                }}
            ]
        }

It doesn't work for basicDate also, but works as expected for simple UTC formats like yyyy/MM/dd. Also it works fine if I put date fields description directly to type mapping, like this:

      "DirectoryRepository_Contact" : {
            "properties" : {
                "properties" : {
                    "type" : "object",
                    "properties" : {
                        "bday": {
                            "type": "date",
                            "format": "basicDateTimeNoMillis"
                        }
                    }
                }
            }
        }

So looks like ISO formats are ignored in dynamic_date_formats setting.

[clintongormley]

Hi @PolinaMash

This is a known issue (see #1694). The problem at the moment is that date-time parsing is not strict enough, so eg 1T could be detected as the year 0001.

We'd like to make this date detection stricter and thus more reliable in 2.0. Here's the issue to track these changes: #10971