Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling big longs/ integers by REST layer #61109

Open
mayya-sharipova opened this issue Aug 13, 2020 · 4 comments
Open

Handling big longs/ integers by REST layer #61109

mayya-sharipova opened this issue Aug 13, 2020 · 4 comments
Labels
>enhancement :Search/Search Search-related issues that do not fall into other categories

Comments

@mayya-sharipova
Copy link
Contributor

Some ES clients can't handle well big integers that ES server may pass to them. For example:

  • ES JS client can't handle well integer number >= 2^53 (For details see comment).
  • Kibana has a similar issue
  • ES PHP client: it supports 64-bit integer but not unsigned. We can support BigInteger using the GMP extensions of PHP (not default in the language).

It would be nice if a client can tell the ES server in what form it expects these big integers to be returned.


As an example, I am adding @ezimuel suggestion here:

  • add a bigint_as_string=1 in the query string (0 by default) to return 64-bit and BigInteger as string; this will prevent JSON decoder to throw errors if numeric > 32-bit.
  • add a numeric_type field in the response that will alert the client for using 64 (bit) or big (as BigInteger). This alert will not have 32 (bit) value, as default. This alert will be triggered if at least one numeric value in the response is 64 or big.
@mayya-sharipova mayya-sharipova added >enhancement needs:triage Requires assignment of a team area label :Search/Search Search-related issues that do not fall into other categories and removed needs:triage Requires assignment of a team area label labels Aug 13, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/Search)

@elasticmachine elasticmachine added the Team:Search Meta label for search team label Aug 13, 2020
@mayya-sharipova mayya-sharipova added :Core/Infra/REST API REST infrastructure and utilities and removed Team:Search Meta label for search team labels Aug 13, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/REST API)

@delvedor
Copy link
Member

It would be nice if a client can tell the ES server in what form it expects these big integers to be returned.

But also how these big integers are sent to Elasticsearch :)
I think that if we go down the path of the bigint_as_string query parameter, ES should both send and accept bigint as strings.

In addition to what we said in #32434, I think that supporting bigint as strings in the rest layer it's important for Elasticseach, as one of its best features is that it's very easy to connect to it with any language/platform. Maintaining this interoperability would allow having enough flexibility to support any use case, and keeping the getting started experience flawlessly. What do you think?

@rjernst rjernst added the needs:triage Requires assignment of a team area label label Dec 3, 2020
@jaymode jaymode added team-discuss and removed needs:triage Requires assignment of a team area label labels Dec 9, 2020
@rjernst
Copy link
Member

rjernst commented Jan 26, 2021

I'm hesitant to have a flag that is supposed to work across the entire REST response. It is really the responsibility of the code writing each section of the response to determine the appropriate format. This seems very similar to me to the print format for date fields, where each individual date field controls how it is printed. That would be a more direct and easy approach to returning bigints as strings since field mappers already have the concept of formatting a value from a field.

@rjernst rjernst removed :Core/Infra/REST API REST infrastructure and utilities team-discuss labels Jan 26, 2021
@elasticmachine elasticmachine removed the Team:Core/Infra Meta label for core/infra team label Jan 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :Search/Search Search-related issues that do not fall into other categories
Projects
None yet
Development

No branches or pull requests

5 participants