New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integer division doesn't work with left operands larger than LONG_MAX #92

Closed
Mischa-Alff opened this Issue Jan 11, 2015 · 2 comments

Comments

Projects
None yet
3 participants
@Mischa-Alff

Mischa-Alff commented Jan 11, 2015

The issue at elm-compiler#878 mentions it, but it's in the wrong repo, so I'm recreating it here.

LHS arguments greater than LONG_MAX(4294967295) return a negative number when using integer division (//).

main = asText (4294967295 // 2)
-- shows 2147483647
main = asText (4294967296 // 2)
-- shows -2147483648

This is because elm-core's src/Native/Basics.js div function is a/b|0, thus using JavaScript's OR operator, which most converts the LHS to a 32-bit integer and returns a 32-bit integer.

EDIT: I read the JS spec, it's not a bug with JS, just another one of its problems.

@int-index

This comment has been minimized.

Show comment
Hide comment
@int-index

int-index Apr 21, 2015

I have encountered this bug and my workaround was to define my own integer division:

(//) : Int -> Int -> Int
a // b = floor (Basics.toFloat a / Basics.toFloat b)

Is there a better implementation?

int-index commented Apr 21, 2015

I have encountered this bug and my workaround was to define my own integer division:

(//) : Int -> Int -> Int
a // b = floor (Basics.toFloat a / Basics.toFloat b)

Is there a better implementation?

@evancz

This comment has been minimized.

Show comment
Hide comment
@evancz

evancz Sep 22, 2016

Member

Consolidated all the math related stuff into the #721 meta issue. Follow along there!

Member

evancz commented Sep 22, 2016

Consolidated all the math related stuff into the #721 meta issue. Follow along there!

@evancz evancz closed this Sep 22, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment