-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix overflow bug in DividePrim #292
Conversation
6768b13
to
2efb95a
Compare
Thanks a lot of the PR, I opened issues on the related repos. Will have a look at it, and let the benchmarks run to get an impression of the impact of this change. Perhaps one idea for an experiment to see how performance is impacted is to do this as a post-computation check. Todos for myself:
|
fd7c46b
to
2bff33a
Compare
@fniephaus I am confused. I added a test, and it passes without your change. diff --git a/core-lib/TestSuite/IntegerTests.ns b/core-lib/TestSuite/IntegerTests.ns
index 5f6aac59f..f95ba7132 100644
--- a/core-lib/TestSuite/IntegerTests.ns
+++ b/core-lib/TestSuite/IntegerTests.ns
@@ -195,6 +195,8 @@ class IntegerTests usingPlatform: platform testFramework: minitest = (
self assert: 3 equals: 7 / 2.0.
self assert: 2 equals: 7 / 3.5.
+
+ self assert: 9223372036854775808 equals: (* Java.MIN_VALUE *) -9223372036854775808 / -1.
)
public testDouble = (
What am I doing wrong? |
`Long.MIN_VALUE / -1` is the only division that overflows. Since the result is well-known, it can be precomputed. "In cases where the size is int or long and overflow errors need to be detected, the methods addExact, subtractExact, multiplyExact, and toIntExact throw an ArithmeticException when the results overflow. For other arithmetic operations such as divide, absolute value, increment, decrement, and negation overflow occurs only with a specific minimum or maximum value and should be checked against the minimum or maximum as appropriate." See https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html Signed-off-by: Stefan Marr <git@stefan-marr.de>
Signed-off-by: Stefan Marr <git@stefan-marr.de>
2bff33a
to
cea6675
Compare
Ok, I figured it out. The first tests literal was directly parsed into BigIntegers, which hid the issue. |
Whoops...fun! :) |
Thanks! Benchmarks look ok. Change seems to work. So, I merged it. |
Long.MIN_VALUE / -1
is the only division that overflows.Since the result is well-known, it can be precomputed.
"In cases where the size is int or long and overflow errors need to be detected, the methods addExact, subtractExact, multiplyExact, and toIntExact throw an ArithmeticException when the results overflow. For other arithmetic operations such as divide, absolute value, increment, decrement, and negation overflow occurs only with a specific minimum or maximum value and should be checked against the minimum or maximum as appropriate."
See https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html