Fix corner case of very large integer parsing that promotes to BigInt #118
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes JuliaData/CSV.jl#1007. The issue here is there's actually a git sha "19129370688824811353f0bcee35b917" where the multithreaded CSV.File case was trying to detect if this value was a float or not. But attempting to call `Parsers.xparse` threw an error, which shouldn't happen from the Parsers.xparse code path (it should just return an error code in the parsing result). The problem was it parsed the very large integer 19129370688824811353, then `f` which it treated as the exponenent character, then `0` as the exponent. When it went to do the scaling code, it promoted the integer to `BigInt`, but the `_scale` BigInt path didn't handle `0` exponent form. This is easily handled before we promote to `BigInt` since we're really just looking to convert the large integer to the desired type.
Codecov Report
@@ Coverage Diff @@
## main #118 +/- ##
==========================================
+ Coverage 87.65% 87.70% +0.04%
==========================================
Files 9 9
Lines 2309 2309
==========================================
+ Hits 2024 2025 +1
+ Misses 285 284 -1
Continue to review full report at Codecov.
|
What's the CSV codepath that's hitting this? Is it in |
|
The latter; |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes JuliaData/CSV.jl#1007. The issue here is there's
actually a git sha "19129370688824811353f0bcee35b917" where the multithreaded
CSV.File case was trying to detect if this value was a float or not. But attempting
to call
Parsers.xparsethrew an error, which shouldn't happen from the Parsers.xparsecode path (it should just return an error code in the parsing result). The problem
was it parsed the very large integer 19129370688824811353, then
fwhich it treatedas the exponenent character, then
0as the exponent. When it went to do the scalingcode, it promoted the integer to
BigInt, but the_scaleBigInt path didn'thandle
0exponent form. This is easily handled before we promote toBigIntsince we're really just looking to convert the large integer to the desired type.