Rounding error on keyword field in Discover

[bczifra]

Kibana version: 6.5-7.1 (at least)

Elasticsearch version: 6.5-7.1 (at least)

Server OS version: ESS

Browser version: Version 74.0.3729.157 (Official Build) (64-bit)

Browser OS version: OSX 10.13.6

Original install method (e.g. download page, yum, from source, etc.): ESS

Describe the bug:
A document with a keyword field that has a value of 9800125870205439 in Elasticsearch is rendered as 9800125870205440 in Kibana's Discover pane.

Steps to reproduce:

DELETE foo

PUT foo
{
  "mappings": {
    "properties": {
      "bar": {
        "type": "keyword"
      }
    }
  },
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  }
}

POST foo/_doc/1
{
  "bar": 9800125870205439
}

GET foo/_search

The /_search returns:

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "foo",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "bar" : 9800125870205439
        }
      }
    ]
  }
}

I then create an index pattern based on foo and accept all the defaults, then go to Discover, select that index pattern, and I see:

Checking Chrome Developer Tools' Network panel I see:

POST https://$id.europe-west1.gcp.cloud.es.io:9243/elasticsearch/_msearch?rest_total_hits_as_int=true&ignore_throttled=true
{"index":"foo*","ignore_unavailable":true,"preference":1559908215939}
{"version":true,"size":500,"sort":[{"_score":{"order":"desc"}}],"_source":{"excludes":[]},"stored_fields":["*"],"script_fields":{},"docvalue_fields":[],"query":{"bool":{"must":[],"filter":[{"match_all":{}}],"should":[],"must_not":[]}},"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}},"fragment_size":2147483647}}

which then results in:

{"took":1,"responses":[{"took":1,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":1,"max_score":0,"hits":[{"_index":"foo","_type":"_doc","_id":"1","_version":1,"_score":0,"_source":{"bar":9800125870205440}}]},"status":200}]}

Note that the value above is the incorrect 9800125870205440, not 9800125870205439.

Everything seems fine in Elasticsearch, or even when querying in DevTools in Kibana. I'm only seeing this happen in Discover.

NOTE: This appears to be some sort of type conversion/rounding issue. Note that if I index the value with quotes, like this:

POST foo/_doc/1
{
  "bar": "9800125870205439"
}

then the value is rendered correctly in Kibana's Discover panel.

Expected behavior:
The document would render with the correct, original value of 9800125870205439.

Errors in browser console (if relevant): None

[elasticmachine]

Pinging @elastic/kibana-app

[timroes]

This will be a very hard one to fix, and comes down a lot to the problems of JavaScript only having 64bit integer numbers (see also #31424 for a more detailed explanation of that problem). The value you specified is above the range that JavaScript still can store numeric values precisely, so we loose something as soon as it becomes a JavaScript number.

Why is it working in Console? In console the response will never be parsed to JSON, but instead we use the xhr.responseText which is the pure string representation of the complete value (which has the correct number) and just print that into the output text box.

But for pretty much all other parts of Kibana we actually use the response as JSON and in that very moment where we're getting that response, meaning in that very moment we lost the precision in the number and we're not having any chance of restoring it again.

In contrast to the date_nanos where we could actually work around with the string format, I don't see how we could solve it here. What puzzles me a bit and just for clarifying that: Elasticsearch returns the _source key as a JSON number, because that was the way it was indexed, even though it was indexed into a keyword (string) field? (@s1monw can answer that most likely). Because if it's the desired behavior that _source contains a numeric JSON value and we don't have a string representation available there is not that much we can do right now to fix this issue.

cc @peterschretlen just so you aware, since that 64bit issue came up already in a couple of different forms.

[flash1293]

Seems like _source is exactly what was passed in during index, no processing at all:
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html#mapping-source-field

The _source field contains the original JSON document body that was passed at index time.

[kertal]

Actually there is a JSON parser library that handles BigInt by converting them to string:
https://github.com/sidorares/json-bigint
It's the follow up of the following stackoverflow:
https://stackoverflow.com/questions/18755125/node-js-is-there-any-proper-way-to-parse-json-with-large-numbers-long-bigint

[s1monw]

In contrast to the date_nanos where we could actually work around with the string format, I don't see how we could solve it here. What puzzles me a bit and just for clarifying that: Elasticsearch returns the _source key as a JSON number, because that was the way it was indexed, even though it was indexed into a keyword (string) field? (@s1monw can answer that most likely). Because if it's the desired behavior that _source contains a numeric JSON value and we don't have a string representation available there is not that much we can do right now to fix this issue.

_source is always returned the same way it's passed to elasticsearch. I don't think you can index _source as a keyword?

[timroes]

I will close this issue since there's nothing we can do in the short term, since JavaScript doesn't support 64bit numbers by default and will instead reference to the overall discussion for 64bit integer values: #1274