You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Hive wrote their own decimal parser. The problem with their implementation is that they do not allow exponents that are above 99 or smaller than -99. Our implementation in spark rapids JNI allows this and does the right thing.
It also has issues with numbers that are too large to fit in 39 digits before the exponent is applied.
Steps/Code to reproduce bug
After #7245 is merged in. Run the tests for "hive-delim-text/extended-float-values" that are against a decimal types. (they should be marked with this issue number)
on the GPU results in 0.0, but on the CPU it is null because the exponent's absolute values is above 99. I think this would work for positive exponents above 99 too, but I don't know how to get a negative scale decimal value into a hive table that spark reads.
Also if the part of the number that is before the radix point is too large (I think it is over 39 digits) that too will fail and return a null on the CPU, but not on the GPU.
1111111111111111111111111111111111111111E-39
Expected behavior
For hive we parse decimal values in the same buggy way as Hive does.
Additional context
I don't think this is critical to fix. I mostly want to document this.
The text was updated successfully, but these errors were encountered:
Describe the bug
Hive wrote their own decimal parser. The problem with their implementation is that they do not allow exponents that are above 99 or smaller than -99. Our implementation in spark rapids JNI allows this and does the right thing.
It also has issues with numbers that are too large to fit in 39 digits before the exponent is applied.
Steps/Code to reproduce bug
After #7245 is merged in. Run the tests for "hive-delim-text/extended-float-values" that are against a decimal types. (they should be marked with this issue number)
on the GPU results in 0.0, but on the CPU it is null because the exponent's absolute values is above 99. I think this would work for positive exponents above 99 too, but I don't know how to get a negative scale decimal value into a hive table that spark reads.
Also if the part of the number that is before the radix point is too large (I think it is over 39 digits) that too will fail and return a null on the CPU, but not on the GPU.
Expected behavior
For hive we parse decimal values in the same buggy way as Hive does.
Additional context
I don't think this is critical to fix. I mostly want to document this.
The text was updated successfully, but these errors were encountered: