Search response - deserialization of numbers (int, double)

[zsonglorman]

Elastic.Clients.Elasticsearch version: 8.9.0

Elasticsearch version: 8.8.1

.NET runtime version: 6.0

Operating system version: Windows 10

Description of the problem including expected versus actual behavior:
Number data types in search response hits source are lost (whether it is a double, int, etc.)

Steps to reproduce:

1. Create index with the following mappings:

"category": {
  "type": "keyword"
},
"availabilityValue": {
  "type": "integer"
},
"dimension": {
  "type": "double"
}

2. Upload some documents
3. Execute search request: var result = client.SearchAsync<Dictionary<string, object>>(request);

In the older NEST client, the response hits source contained all fields in their respective data types, so it was easy to use them i.e.:

result.Hits.First().Source["category"].GetType().Name -> String
result.Hits.First().Source["availabilityValue"].GetType().Name -> Int32
result.Hits.First().Source["dimension"].GetType().Name -> Double

However, in the new Elastic.Clients.Elasticsearch client, these hit sources are JsonElements, where type of number is lost:
result.Hits.First().Source["category"] -> ValueKind = String
result.Hits.First().Source["availabilityValue"] -> ValueKind = Number
result.Hits.First().Source["dimension"] -> ValueKind = Number

For the 2 numbers, the information whether it was originally a double or int is lost, so we are unable to decide whether to call GetInt32 or GetDouble on the json element to get its value.

I'm not sure whether it's a bug, or there's a solution to our problem that we are not aware of.
Thank you in advance!

[flobernd]

Hi there,

this is a difference of how the old JSON library vs. the new System.Text.Json library handles deserialization to object. The JSON spec itself only specifies number. There is no differentiation between int, float or even int/int[32] and long/int[64].

It is possible to re-implement the old behavior by registering a custom source JsonConverter like e.g. described here:
https://stackoverflow.com/a/65974452

If you ask me personally, I would probably not go that route but instead cache the schema/mappings on application start (by using the GetMappings API) for best performance.

Another option would be to use TryGetInt32, TryGetFloat, and so on in sequence until one of the call suceeds. This is basically what the old JSON library did as well.