Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot index document whose date value is equivalent to Java's OffsetDateTime.MIN #81042

Open
awalter17 opened this issue Nov 25, 2021 · 2 comments
Labels
>enhancement :Search/Mapping Index mappings, including merging and defining field types Team:Search Meta label for search team

Comments

@awalter17
Copy link

Elasticsearch version (bin/elasticsearch --version): Version: 7.10.2, Build: oss/docker/747e1cc71def077253878a59143c1f785afa92b9/2021-01-13T00:42:12.435326Z, JVM: 15.0.1

Plugins installed: []

JVM version (java -version): openjdk 11.0.13 2021-10-19 <- that's the java version on my machine not in the docker container

OS version (uname -a if on a Unix-like system): Linux, Fedora 34

Description of the problem including expected versus actual behavior:

It appears as though elasticsearch cannot index a date whose value is equivalent to OffsetDateTime.MIN. When I try to index a document with such a date field, I get a 400 response with an arithmetic_exception (full response in steps below).

The date String -999999999-01-01T00:00:00+18:00 comes from a Java snippet like this:

OffsetDateTime min = OffsetDateTime.MIN;
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("uuuu-MM-dd'T'HH:mm:ssxxx");
String minString = min.format(formatter);
System.out.println(minString);

Steps to reproduce:

Please include a minimal but complete recreation of the problem,
including (e.g.) index creation, mappings, settings, query etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

  1. Start ES docker container
    docker run -p 9200:9200 -p 9300:9300 -e discovery.type=single-node docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
  2. Create index with date field, PUT to http://localhost:9200/testing-date with JSON body
{
   "settings" : {},
   "mappings" : {
       "properties" : {
           "test_date" : {
               "type" : "date",
               "format" : "uuuu-MM-dd'T'HH:mm:ssxxx"
           }
       }
   }
}

Response:

{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "testing-date"
}
  1. Add document to that index whose date is a string representaiton of Java's OffsetDateTime.MIN, POST to http://localhost:9200/testing-date/_doc with JSON body:
{
    "test_date" : "-999999999-01-01T00:00:00+18:00"
}

Response:

{
    "error": {
        "root_cause": [
            {
                "type": "mapper_parsing_exception",
                "reason": "failed to parse field [test_date] of type [date] in document with id 'zG_jVn0Buy9QhGs3zL4S'. Preview of field's value: '-999999999-01-01T00:00:00+18:00'"
            }
        ],
        "type": "mapper_parsing_exception",
        "reason": "failed to parse field [test_date] of type [date] in document with id 'zG_jVn0Buy9QhGs3zL4S'. Preview of field's value: '-999999999-01-01T00:00:00+18:00'",
        "caused_by": {
            "type": "arithmetic_exception",
            "reason": "long overflow"
        }
    },
    "status": 400
}

Provide logs (if relevant):

Please let me know if you require any additional information from me.

@awalter17 awalter17 added >bug needs:triage Requires assignment of a team area label labels Nov 25, 2021
@albertzaharovits albertzaharovits added the :Search/Mapping Index mappings, including merging and defining field types label Nov 28, 2021
@elasticmachine elasticmachine added the Team:Search Meta label for search team label Nov 28, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@albertzaharovits albertzaharovits removed the needs:triage Requires assignment of a team area label label Nov 28, 2021
@ywelsch ywelsch added >enhancement and removed >bug labels Nov 29, 2021
@ywelsch
Copy link
Contributor

ywelsch commented Nov 29, 2021

Dates in Elasticsearch are internally stored as a single long number representing milliseconds-since-the-epoch, as also documented here: https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html

OffsetDateTime.MIN cannot be represented with a single long value (it's too far back), hence the following failing: OffsetDateTime.MIN.toInstant().toEpochMilli(); // throws java.lang.ArithmeticException: long overflow

Instants can't be represented with a single long value, see Javadocs of Java's Instant class:

The range of an instant requires the storage of a number larger than a long. To achieve this, the class stores a long representing epoch-seconds and an int representing nanosecond-of-second, which will always be between 0 and 999,999,999. The epoch-seconds are measured from the standard Java epoch of 1970-01-01T00:00:00Z where instants after the epoch have positive values, and earlier instants have negative values. For both the epoch-second and nanosecond parts, a larger value is always later on the time-line than a smaller value.

I think that the limitation of not being able to represent such dates is ok, the exception message could be clearer, however, and so I'm keeping this issue open as an enhancement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :Search/Mapping Index mappings, including merging and defining field types Team:Search Meta label for search team
Projects
None yet
Development

No branches or pull requests

4 participants