Add new mathematical operators #2887

Merged
merged 10 commits into from Jan 24, 2014

Projects

None yet
@epidemian

This is an attempt to (finally) include a new set of mathematical operators beyond what JavaScript provides.

Included on this PR are the operators that seemed the most consensually accepted by the community and that seemed the most practical and intuitive in terms of syntax to me:

  • the power operator **,
  • the floor division operator // and
  • the (correct) modulo operator %%.

Other operators might be useful, but i think it's important not to over do it and try to convert every function on Math into a new operator. So i deliberately left out more sophisticated operators like LiveScript's chainable maximum and minimum operators (<? and >?) or a unicode operator for square roots 😸

That being said, this is an open pull request, so if you think other operators deserve to make it in (or some of these doesn't deserve it), please comment.


Small caveat

The floor division operator token clashes with the current empty regex token: //. I think there's no way to have them both; even if we did some grotesque lexer + parser magic introducing some ambiguity in the lexer (say, a FLOOR_DIV_OR_EMPTY_REGEX token) and then trying to disambiguate it in the parser, there are ugly corner cases, like a //i, which is valid on the current master as a call to a with an empty regex and is also valid in this PR as a floor division between a and i.

I opted to remove the support for empty regexes in this PR (notice a missing regex test), but i don't know if this is an acceptable removal. I would expect that nobody would ever use an empty regex, as i see no point in doing so, but i've been very mistaken about guessing what other people don't do on their code in the past. So, if we decide to remove the empty regex token, could we make an intermediate minor release that marks that token as a syntax error and some time after that release these new set of operators?

@epidemian

Related stuff: #1971, #2026

@vendethiel
Collaborator

a //i, which is valid on the current master as a call to a with an empty regex and is also valid in this PR as a floor division between a and i

This case could be fixed by requiring it to be either unspaced or dually-spaced, couldn't it ?

@epidemian

This case could be fixed by requiring it to be either unspaced or dually-spaced, couldn't it ?

Hmm... yes, but not without other nuances i think. When other operators appear at the end of a line they imply a line continuation:

# Parsed as "foo(bar + baz);"
foo bar +
baz

But with the // token as a regex, this has a valid meaning right now:

# Parsed as "foo(bar(/(?:)/)); baz;"
foo bar //
baz

(it seems the // also confuses the GitHub syntax highlighter too 😺 )

We could also opt for another symbol for floor division; though i would prefer it to be the same symbol as in Python.

@bjmiller

I think the "easy" way to solve this is just use a different operator. Yes, "//" is the most make-sensical, and it's already well-known from python, but it saves a lot of effort to just use something different. I'll throw "/_" ("div-floor") out there, although it's neither better nor worse than most other alternatives.

@shesek

@bjmiller since _ is a valid javascript identifier, a /_ b is already parsed as a / _(b)

@michaelficarra
Collaborator

@shesek and a/_b is already parsed as a / _b.

@epidemian

Sooo... any other thoughts on this PR? Any other operator that would be useful?


What should we do with the // empty regex literal? If the // is preferred for the floor division operator we could release a new compiler error in the next version of CS to avoid changing the meaning of users' code unexpectedly. Something like:

$ bin/coffee -cs <<<'a // b'
[stdin]:1:3: error: // reserved for future use as floor division operator. Use /(?:)/ for an empty regex.
a // b
  ^

Then, after that release, we could finally add the new shiny // operator and hopefully avoid having broken many users' code.


Also, about the documentation, @jashkenas is good with the words, but maybe we can help providing some sample snippets for the new operators (bonus point for a coherent example that showcases all three of them 😸). I may fiddle with this later if i feel inspired =P

@bjmiller

Just double-checking: Is there a real use case for an empty regex? Like, why would anyone intentionally create one?

Anyway, it's still my preference to come up with an operator that requires less work to implement. The problem is, I don't really see any other combination of symbols on the keyboard that make any kind of mnemonic sense to me.

@epidemian

Just double-checking: Is there a real use case for an empty regex? Like, why would anyone intentionally create one?

I have no idea. The /(?:)/ is always there too, which is what a REPL will yield if you type new RegExp :)

Anyway, it's still my preference to come up with an operator that requires less work to implement.

Implementing // as an operator is very easy actually. The only change regarding the // regex is this line here (instead of substituting it with /(?:)/ it returns an empty match, thus allowing // to be an operator).


About the documentation for the new operators, maybe instead of examples we can have a simple table showing the JS equivalents:

CoffeeScript JavaScript equivalent
Power a ** b Math.pow(a, b)
Floor division a // b Math.floor(a / b)
True modulo a %% b (a % b + b) % b

Though, to be in tone with the rest of the documentation maybe some runnable examples would be better, i don't know. Some ideas:

# For the modulo operator, maybe a circular array index.
arrAt = (arr, i) -> arr[i %% arr.length]
notes = ['do', 're', 'mi', 'fa', 'sol', 'la', 'si']
alert "The note before do is #{arrAt notes, -1}"
alert "And 7 notes after do is #{arrAt notes, 7} again!"

# For the floor division, any division that should be rounded would be good IMO.
seconds = (new Date - start) // 1000
alert "You have been reading the docs for #{seconds} seconds."
# (`start` could be taken when the page is loaded)

# For the power operator, maybe a statistical formula like exponential growth.
population = 7000
rate = 0.05
time = 10
finalPopulation = population * (1 + rate) ** time
alert "Population after 10 years: #{Math.floor endPopulation}"

# Not sure if a good example for the power operator, but i like it =P
φ = 1.618033988749895
ψ = -0.6180339887498949
# A fast fiboncci number formula.
fastFib = (n) ->** n - ψ ** n) /- ψ)
alert "fib(8) = #{fastFib 8}"

I think i would prefer the table for succinctness' sake though.

(totally off-topic: why does the GH highlighter discriminate the awesome φ and ψ symbols with that ugly red background!?)

@michaelficarra
Collaborator

why does the GH highlighter discriminate the awesome φ and ψ symbols with that ugly red background!?

Because pygments sucks. It has one job, and it can never get it right. At least it no longer thinks herecomments never end.

@erisdev

@epidemian pygments believes non-ASCII characters are not valid in CoffeeScript (and presumably many other languages) source. a unicode travesty.

@vendethiel
Collaborator

So long you don't turn Coffee into APL :p.

@lydell
Collaborator

Perhaps not so related to this PR, but when I think about maths and CoffeeScript, this is what comes up in my mind:

Maths and CoffeeScript teacher: “What is the result of sin π + 5?”
Student: “That depends on what you're currently teaching.” (Assuming {sin} = Math; π = Math.PI)

@michaelficarra
Collaborator

@lydell: Agreed, the juxtaposition operator (implicit function application) should have higher precedence than all other infix operators. But that's a discussion for a different issue.

edit: Also, you can do renaming in destructuring assignment: {sin, PI: π} = Math.

@epidemian

Thanks for clarifying, @michaelficarra, i wasn't sure what @lydell was pointing out.


I'd appreciate if we could have some closure for this issue. Should i invoke @jashkenas? :)

Jeremy: do you think this PR is acceptable? What do you think we should do about the clash between the floor operator and the empty regex (//)?

@michaelficarra
Collaborator

@epidemian: I believe we were all happy about the power operator and floored division, but the max/min (which I'm strongly for) and fixed modulo operators were still undecided.

@epidemian

@michaelficarra, thanks for the feedback.

For the min/max operators, i guess you're referring to the LiveScript's cute <? and >? (please correct me if i'm wrong).

For those who do not know, they are basically the operator form of Math.min and Math.max respectively. And they are chainable! So a <? b <? c is equivalent to Math.min a, b, c; though it probably should be compiled to the more hand-rolled (_ref = a < b ? a : b) < c ? _ref : c form, as they do on LiveScript, to allow for comparison of strings for example.

I personally find these two operators to be quite intuitive and consistent with current binary ? operator: a <? b can be read as "if a is less than b then a else b" in the same way that a ? b can be read as "if a is defined then a else b".

That being said, the other operators included in this PR were chosen mostly for their "familiarity" in other languages as well as their convenience. In the case of **, it was chosen mostly for the familiarity factor, as using Math.pow instead is not that big of a deal, but programmers coming from Python or Ruby (both of which could be considered Coffee's ancestors) as well as other languages might expect a power operator instead. // was chosen mostly for it's convenience, as floored division is kind of the norm when dealing with integer numbers. And %% was chosen for a mixture of both familiarity and convenience: lots of other languages implement modulo as the proposed %% instead of JS's %, and being able to write a %% b instead of remembering the (a % b + b) % b formula is pretty convenient too.

In the case of <? and >?, i don't think they are so decisively convenient over using Math.max/min, and i feel they are not as popular in other programming languages either. Besides, we should always consider the extra complexity of adding new operators, which is mostly adding one more case to the precedence rules.

All in all, my vote for the inclusion of these two operators is neutral. The popularity of the operators in other languages doesn't really bother me that much (chained comparisons are not that popular outside Coffee either, Python being the only exception that comes to mind, and they're damn useful for example). I'll not be sad if they are rejected, but i will happily implement them if they are accepted 😺

I'm interested in what other people think.

@vendethiel
Collaborator

I don't think I've used coco's <? ?> More than 4 times in a year, so I don't dind them that attractive

@satyr
Collaborator

to allow for comparison of strings for example

They're mostly for strings to begin with, as you should be using Math.{min,max} for numbers. I don't consider them mathematical operators.

@L8D

Can I please squeeze the statement that // needs to be usable for standalone things like // Math.random() * 2 becomes Math.floor(Math.random() * 2)(Because ~~ in many contexts is broken). And also possibly a symbol for Math.random() like ?? or ????

I find my self tediously typing Math.floor and Math.random way too many times.

Also, please don't forget the shuttle operator which is just as useful. a <=> b -> a < b ? -1 : a > b ? 1 : 0

@loveencounterflow

some thoughts:

(1) i like how min, max behave in Python;
basically, if you say min x, you'll get the equivalent of min x.... regardless of whether you call min
x, f
or min a, b, c, f, the last argument, if callable, will be used to determine sorting. unlike JS,
you are not allowed to say min() with no arguments, which makes sense. that said, people here will
probably not want to mess with the signature of (the CS equivalent of) a JS built-in, which is a strong
point. one could also say that reading min x as though it was written min x... involves too much magic,
and that the argument splatting should always be done explicitly. we can already do all of the above in a
library, so maybe let's not touch Math.min, Math.max.

(2) some people argue in favor of a >? b, a <? b. three objections. (a) this is a binary
operator, right? so what am i gonna do if i want to have the smallest among three values? a <? b <? c?
does that make sense? is it anything better than min a, b, c? i doubt that. how do you write 'the smallest
value in x'? like, <?.apply x ?? (joking). it is not obvious. (the same applies to transforming a + b
into sum of all values in x).

(2b) the ? in <? is clashing with the lovely and important existence operator. frankly, ? is a brilliant syntactical sleight-of-hand—its rules are so useful and logical that when i started using it, i often just thought, well, default-assignment should be written, like, target = d[ 'name' ]?= {}, and it just worked. the syntactics of ? are pretty involved when you start to think of it, and IMHO introducing
>?-style operators would threaten the cohesion of those rules. get prepared for hedges like '...except in >?, <?,
where the question mark is not the existential operator' in the docs. -1.

(2c) people have argued in favor of, IIRC, , for min and max, and for sqrt. i'd say, if
that's good for code clarity in your module, fine, just go ahead and preface it with ⋁ = min, ⋀ =
max
, √ = sqrt, just great (except see below). i'm all for 'embracing unicode' where it makes sense.
mathematical papers seem often to rely on fixing a particular notation for the purpose in the front matter.
but forcing those symbols onto the broad audience? not such a good idea. it's a shame there is no one widely
accepted, internationally keyboard-accessible symbol for Math.pow, so we get all of pow, ^, ** in
current languages, where would have arguable gave been best: x = 2 ↑ 3, perfect. but force the arrow
as the only way to have exponentiation in CS now? not so good.

the thing here is that if you have a function call, you can already rename that function, and you'll get
8 == ↑ 2, 3 today—locally, in your module (to be true, this sadly does not work, as is classed a
symbol in JS. i'd argue that CS should allow this usage and just translate those non-letters in variable
names).

BUT if you absolutely insist that you must write x = 2 ↑ 3, you can't do that without changing
the language, and then you change it for every single user of that language. operators cannot be introduced
or re-defined
. my impression is that many discussions in languages like Python and CS that revolve around
not introducing new compulsory syntax for all users and all the legacy code. if you're watching that
limping (non-) transition from Py2 to Py3, you know what i'm talking about.

so ideally, we'd have a language that you can modify locally, for the needs of your particular module.
individual modules could opt-in or opt-out of particular features, and modify syntax to a degree. i'm
not saying it's sensible to start and do that right now, with the current version of CS—it is not
suitable for this kind of mallable syntax. i believe it can be done, especially when using composable
parser combinators (some slides; hit space to advance to next slide).

the outcome: i'd prefer the letter-named functions min, max, maybe even pow, floordiv, (divmod
anyone?) over many of the proposed symbol-named operators: they are more keyboard-accessible; they are
more readable to the 'casual' reader; they are practically guaranteed not to introduce new hard-to-deal-with
edge cases into the grammar; they may be aliased where need be (at least greek letters and the like do work
today out-of-the-box); and in some future or some parallel universe, you'll still be able to transform them
into symbol-operators.

footnote i love both re = /[0-9]+/ and x = a // b, but /.../ is already in for RegExes and does
have its own set of oopses (like you cannot start a RegEx literal with a space, must escape slash...), so
better not mess with it just for the sake of making floordiv a, b more cute. isn't c = floor a / b already
there and working? not good enough?

@loveencounterflow

ah and +1 for a <=> b. gotta love it.

@vendethiel
Collaborator

<=> was turned down in #2672

@loveencounterflow

rightly so, having read the comments on that one. if we could just <=> = make_binary_operator ( left, right ) -> ... locally, that would be great.

@xixixao

@loveencounterflow I don't agree with your conclusion for max, min, pow... to be added - precisely for the reason you stated, you can always declare them yourself at the top of the file. It only makes sense to talk about operators here, infix operators at that, because functions are already available otherwise.

Could we get the (I think from what I read in the many threads) widely acceptable operators //, **, %% in? I think getting rid of // regexp is a small price to pay (still haven't seen a legitimate/frequent use case for it). The max and min operators seem just too magical, and it would be sad if we got caught up on them.

@davidchambers

Could we get the (I think from what I read in the many threads) widely acceptable operators //, **, %% in?

That would be wonderful.

I think getting rid of // regexp is a small price to pay (still haven't seen a legitimate/frequent use case for it).

Agreed.

The max and min operators seem just too magical, and it would be sad if we got caught up on them.

Agreed.

@xixixao xixixao commented on the diff Nov 26, 2013
test/operators.coffee
@@ -296,3 +296,58 @@ test "#2567: Optimization of negated existential produces correct result", ->
test "#2508: Existential access of the prototype", ->
eq NonExistent?::nothing, undefined
ok Object?::toString
+
+test "power operator", ->
+ eq 27, 3 ** 3
@xixixao
xixixao Nov 26, 2013

Question (off-topic): The eq error message says:
"Expected left to equal right". Let's say this test breaks: Expected 27 to equal 14, I would read it as 27 being wrong. I noticed that the use of eq isn't consistent in the test suite, but it would be great if it was, so that the errors were more readable. (I know that this isn't completely sensible, the message should really say Expected 3 ** 3 to equal 27, got 14 instead.)

@epidemian
epidemian Nov 27, 2013

Yeah, you're right about that error message. The rest of this file seems to be using eq <expected>, <actual> though. I'd recommend to swap the wrong calls to eq in a separate refactor-only PR.

@xixixao
xixixao Nov 27, 2013

Sounds good.

@L8D
@vendethiel
Collaborator

Not in coffeescript.

This was referenced Nov 28, 2013
@jashkenas
Owner

@xixixao I think probably go for it. My two questions are:

Does the modulo operator really need a helper function, instead of being compiled inline?

What are some good real-world use cases for the modulo operator, where the existing remainder operator % won't suffice?

@xixixao

Many cases, I have written the following pattern way too many times in my life:

i = something
array[(i + array.length) % array.length]

(notice that's not even bulletproof). +1 for inline compilation, @epidemian ?

@jashkenas
Owner

I have written the following pattern way too many times in my life

That's exactly what I mean — you're just using the remainder operator there, not @epidemian's proposed "correct" modulo operator. Where does the correct one work that the remainder won't help you?

@xixixao

@jashkenas My code is error prone and longer than

i = something
array[i %% array.length]

Now the correct modulo not only fixes the case of negative left operand but also negative right operand.

@jashkenas
Owner

Why would you have a negative RHS?

@xixixao

Why not? I mean that is not applicable to my example, but it is likely that some algorithm might depend on modulo with negative divisor. If not, we can compile to the more relaxed version.

@xixixao

Original discussion on modulo starts with your comment here: #1971 (comment)

@jashkenas
Owner

it is likely that some algorithm might depend on modulo with negative divisor

That's exactly what I'm asking. Have any of you ever needed "true" modulo for real-world programming before? If so, what for? If not — we can just stick with %, and leave things simple, no?

@epidemian

@jashkenas,

Does the modulo operator really need a helper function, instead of being compiled inline?

Nope, it's just for convenience/laziness: a separate function makes it very easy to guarantee that each operator is evaluated only once. If we were to compile this in-line, we should assign the value the second operand (if it isn't a simple expressions) to an intermediate variable, because it's used more than once in the operation.

A bit more complex, but yes, there should be no problem in doing so. I personally prefer having a separate helper function, as i think _modulo(42, foo()) is much more obvious in its meaning than (42 % (_ref = foo()) + _ref) % _ref. If someone wants to take a stab at compiling it inline, go for it; i may take a look at it tonight.

What are some good real-world use cases for the modulo operator, where the existing remainder operator % won't suffice?

Anything that "wraps around" and may be negative. For example, a "neighbours" function in a Game of Life (stolen from the first article i found...):

neighbours = ([x, y]) ->
  x0 = (x - 1) %% N
  x1 = x
  x2 = (x + 1) %% N
  y0 = (y - 1) %% N
  y1 = y
  y2 = (y + 1) %% N
  [[x1, y0], [x2, y0], [x2, y1], [x2, y2], [x1, y2], [x0, y2], [x0, y1], [x0, y0]]

These kind of operations are pretty common in games :)

If you would prefer to keep this operator out, i'd totally understand. I'm not sure either if having a correct modulo operation is worth the complexity of having a new operator.

@jashkenas
Owner

@epidemian Great example, let's leave it in then.

And your argument about having it as a helper is persuasive as well. Sounds just fine. Feel free to fix up this PR so that it applies cleanly, and then merge away.

@satyr
Collaborator

Make sure to force numbers in the modulo helper. You'd get nonsenses like:

'1' %% '42' == 16
@L8D

@satyr There is no way to effectively type check what a variable would be other than in their literal form. If you want to prevent them from being used in literal form, I really don't think there will ever be a case where someone would need/want that.

@xixixao

@satyr '1' % '42' gives 1 in CS, this is up to the user.

@lydell
Collaborator

I think it’s worth fixing @satyr’s issue. We only need to add two +:

 function(a, b) { return (a % +b + +b) % b; }

That makes %% behave like all other mathematical operators: Using numbery strings as operands gives the same result as using numbers.

@xixixao therefore '1' %% '42' should give 1 as well.

@xixixao

You only need the second + I believe. Otherwise it makes sense. But I disagree with

all other mathematical operators: Using numbery strings as operands gives the same result as using numbers.

coffee> '1' + '42'
'142'

The lesson being you should really not use strings instead of numbers.

@epidemian epidemian Merge branch 'master' into more-math-operators
Conflicts:
	lib/coffee-script/grammar.js
	lib/coffee-script/lexer.js
	lib/coffee-script/nodes.js
	lib/coffee-script/parser.js
	test/regexps.coffee
2b4421f
@epidemian

PR should be mergeable now :)

Should i add @lydell's trick to convert the arguments into numbers (using @xixixao's suggestion)?

function(a, b) { return (a % b + +b) % b; }
@michaelficarra
Collaborator

@epidemian: Yes, please.

@epidemian

Done.

(555 tests!)

@michaelficarra michaelficarra and 2 others commented on an outdated diff Jan 24, 2014
test/operators.coffee
+
+test "floor division operator", ->
+ eq 2, 7 // 3
+ eq -3, -7 // 3
+ eq NaN, 0 // 0
+
+test "floor division operator compound assignment", ->
+ a = 7
+ a //= 2
+ eq 3, a
+
+test "modulo operator", ->
+ check = (a, b, expected) ->
+ res = a %% b
+ # Don't use eq because it treats 0 as different to -0.
+ ok res == expected or isNaN(res) and isNaN(expected),
@michaelficarra
michaelficarra Jan 24, 2014

Don't use == either. Why can't we distinguish 0 from -0 in the tests?

@vendethiel
vendethiel Jan 24, 2014

== is the same as is, let's go with the latter :)

@epidemian
epidemian Jan 24, 2014

@michaelficarra, yeah, the only curious case is 0 %% -1, which evaluates to -0 (which makes sense since the result of the modulus should have the same sign as the divisor), so we could assert that in the tests and be explicit 👍

@lydell
Collaborator

@xixixao oops, all other mathematical operators except + (which is the root of that issue). You’re right, + is the only exception.

@michaelficarra michaelficarra merged commit d687d52 into jashkenas:master Jan 24, 2014
@satyr satyr added a commit to satyr/coco that referenced this pull request Feb 7, 2014
@satyr satyr picked %% from jashkenas/coffeescript#2887 2bf1c7d
@dandv dandv referenced this pull request Feb 22, 2014
Closed

INVALID issue #3385

@deathcap deathcap referenced this pull request in SpigotMC/Spigot Mar 27, 2014
@Thinkofname Thinkofname Handle case where currentTick could be negative
@Aikar is there a better way to handle this?
fdfc07b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment