Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integer overflow in exponent #62

Open
ksvanhorn opened this issue Apr 17, 2018 · 2 comments
Open

Integer overflow in exponent #62

ksvanhorn opened this issue Apr 17, 2018 · 2 comments

Comments

@ksvanhorn
Copy link

@ksvanhorn ksvanhorn commented Apr 17, 2018

scientific 1 (maxBound::Int) * scientific 1 (maxBound::Int) evaluates to 1.0e-2.

@basvandijk

This comment has been minimized.

Copy link
Owner

@basvandijk basvandijk commented Apr 17, 2018

Yes, this is unfortunate. Scientific uses an Int to represent the exponent so it suffers from the same integer overflows that Int suffers from. I've an old TODO to switch the exponent to Integer to make Scientific truly arbitrary precision.

@basvandijk

This comment has been minimized.

Copy link
Owner

@basvandijk basvandijk commented Apr 17, 2018

I started doing some work on this on the integer-exponent branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.