Definition of modulo is broken

[waldemarhorwat]

The change to explicitly note mathematical values broke the definition of modulo in the spec:

The notation “x modulo t y” (y must be finite and nonzero) computes a value k of the same sign as y (or zero) such that abs t(k) <t abs t(y) and x -t k = q ×_t_ y for some integer t q.

where t is either ℝ for mathematical numbers or 𝔽 for IEEE doubles.

This definition of modulo works only when t is ℝ. The formula it uses is not meant to be used with numbers that are not mathematical numbers and breaks badly when t is 𝔽. There are multiple issues with finite precision and rounding, overflow, and underflow.

For example, this definition produces the nonsensical result that the exact IEEE double 200000000000000032𝔽 modulo𝔽 100𝔽 is, among other things, 19𝔽 because 200000000000000032𝔽 -𝔽 19𝔽 = 2000000000000000𝔽 ×𝔽 100𝔽. The correct answer, of course, is 32𝔽.

Similarly, by this definition, 4𝔽 modulo 2𝔽 evaluates to, among other things, 1.74e-75𝔽 because 4𝔽 -𝔽 1.74e-75𝔽 = 2𝔽 ×𝔽 2𝔽.

To fix it, use the formula only for the mathematical definition of modulo. If you need to define modulo on 𝔽, do it by converting to and from mathematical numbers.

[michaelficarra]

@waldemarhorwat Would you be willing to send a PR with your suggested fix?

[ljharb]

cc @littledan @caiolima

[waldemarhorwat]

@michaelficarra will do. It's a bit tricky because I don't want to define it for things like ±∞ and NaN so I need to go through every point of use to see if those can arise. There are also some issues with the definition of abs in this section.

[syg]

Looking forward to seeing the PR for this.

[ljharb]

For reference, some places in ES2019 where modulo is used on Number values:

• https://www.ecma-international.org/ecma-262/10.0/#sec-toindex
• https://www.ecma-international.org/ecma-262/10.0/#sec-makeday
• https://www.ecma-international.org/ecma-262/10.0/#sec-typedarray-buffer-byteoffset-length

[waldemarhorwat]

Having seen so many other bugs introduced by #1135, I've now come to the conclusion that, rather than trying to hack modulo, exponentiation, addition, subtraction, counters, etc., the best way to fix them is to revert #1135 and instead explicitly mark the places where Number or Decimal arithmetic should be used. See #1964.

[littledan]

It seems like the fix for this issue is quite simple, to make modulo only in terms of ℝ and disallow parameterizing it. Are there any downsides to this fix?

[waldemarhorwat]

@littledan: Yes, that's what I'd recommend doing for modulo (and exponentiation, etc.).

[natedogith1]

Modulo also seems inconsistent when dealing with negatives. Mathematical Operations specifies that the result has the sign of the divisor, while Number::remainder specifies that the result has the sign of the dividend.

[jmdyck]

So modulo behaves one way and Number::remainder behaves another. That's not really an inconsistency, not in the sense of the spec contradicting itself.