-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[bug] Inconsistent data type (int/float) within single field/column in results #8707
Comments
@bbczeuz the output is being emitted as expected. All values you're seeing are InfluxDB |
Why close the issue? influxDB cannot be fed back its own results.
I was thinking of a more flexible input parser, but didn't come to a solution - the field type is defined the first time data arrives. If some field was '0', the field will be interpreted as int and following writes to this field will fail if the data is not int. We could send the results of 'SHOW FIELD KEYS' on startup of the sender, but that's ugly - we would need to send dummy data, then delete it again. In addition, this is very unflexible as the sender has to keep track of what schemes exist in his and the receiver's DB (the receiver cannot send requests to the sender). |
I have just run into the same issue in one of my applications. InfluxDB output JSON is showing floats as integers and that is a big issue, IMHO. As @bbczeuz said, data fetched from InfluxDB can't be fed written back to the same collection as that will cause column type issues and result in a HTTP 400 error. As @bbczeuz, my current workaround is also to use Here's a sample of the JSON I am getting:
And here's what I believe I should be getting:
And this is what
@e-dard While you are right about the behavior of Javascript JSON libraries (haven't checked Go), that behavior is problematic nevertheless. In Python, the default JSON library does include decimals when parsing integer floats:
Parsing that list back and forth between JSON and Python does not cause any type inconsistency issues, so I don't see how that is not an issue in a database. |
@gusutabopb @bbczeuz sorry, I should have said that we have a proposal in place to remedy this by moving to an alternative output format to JSON: #8330 I think the only current solution is, as you explained @gusutabopb to check the type first, or have some client-based error handling to detect the wrong type and insert accordingly. |
influxdata/influxdb-python#665 is caused by this issue.
This is not the case of python, for instance: In [1]: import json
In [2]: json.dumps(1)
Out[2]: '1'
In [3]: json.dumps(1.0)
Out[3]: '1.0' @e-dard : This behavior is the cause of many subtle bugs that are hard to debug. The workaround involves doubling the number of requests to the server. Fixing the bug would have no impact on json parsers that do not make a difference between |
Bug report
System info:
Steps to reproduce:
Expected behavior:
One field shall always have the same data type
Actual behavior:
If a float-field value is zero, it's returned as '0' instead of '0.0'. Re-Importing this data to InfluxDB leads to a field type conflict: e.g.
400: {"error":"partial write: field type conflict: input field \"usage_user\" on measurement \"cpu\" is type integer, already exists as type float dropped=1"}
SELECT usage_user::integer FROM telegraf.raw.cpu
: all results arrive as integersSELECT usage_user::float FROM telegraf.raw.cpu
: some results arrive as ints, some as floatsSELECT usage_user + 0.0 FROM telegraf.raw.cpu
: some results arrive as ints, some as floatsSELECT usage_user + 0.5 FROM telegraf.raw.cpu
: all results arrive as floatsSELECT usage_user + 1.0 FROM telegraf.raw.cpu
: some results arrive as ints, some as floatsaa=json.loads('{"bla":0.0, "ble":0}')
Additional info:
The text was updated successfully, but these errors were encountered: