Custom operators

[disnet]

Custom (honu-style) operator support. Still need to do a little bit of cleaning/testing before I merge. Here's the syntax I'm going with at the moment:

unaryop neg 14 {$op} => #{ -$op }
binaryop plus 12 left { $lhs, $rhs } => #{ $lhs + $rhs }

The definition syntax is actually just a macro that desugars down to a primitive form:

binaryop (plus) 12 left {
    macro {
        rule { ($left:expr) ($right:expr) } => {
            $left + $right
        }
    }
}

One interesting question came up while I was working on this. How should we order precedence? The precedence table on mdn is ordered from highest (0) to lowest (18), which seems a bit backwards since bigger numbers have lower precedence. I've flipped this for sweet.js so 0 binds least tightly and 18 binds most tightly. Is this the right choice or will people be confused because of the mdn page?

[natefaubion]

+1 on the precedence change. I think the numbering on the page is just to order it: numbers that come first bind first. Honestly, I think most people who want to use custom operators are coming from languages that support them already, and it'd be more confusing to do it like the MDN page.

[natefaubion]

https://github.com/mozilla/sweet.js/blob/f2219ab22ebaf0137de971d65cb4dc276c8ec987/macros/stxcase.js#L743 throws an ellipsis error for $name.

[natefaubion]

It's also having a hard time with multi-token names comprised of operators:

binaryop (|>) 0 left { $left, $right } => #{ $right($left) }
var foo = 1 |> bar |> baz;

Seems to just strip the operators. Using something else works fine.

[disnet]

Thanks for catching that. Multi token operators should be working now.

[natefaubion]

Grouping with parens results in strange errors:

binaryop (|>) 0 left { $lhs, $rhs } => #{ $rhs($lhs) }
var a = foo |> (bar |> baz);

Error: Line 2: Unexpected identifier
[... ( bar baz ) ) ( ...]
    at throwError (/Users/nathan/Sites/git/sweet.js/lib/parser.js:1887:21)
    at throwUnexpected (/Users/nathan/Sites/git/sweet.js/lib/parser.js:1923:13)
    at expect (/Users/nathan/Sites/git/sweet.js/lib/parser.js:1946:13)
    at trackGroupExpression (/Users/nathan/Sites/git/sweet.js/lib/parser.js:4196:9)
    at parsePrimaryExpression (/Users/nathan/Sites/git/sweet.js/lib/parser.js:2287:20)
    at /Users/nathan/Sites/git/sweet.js/lib/parser.js:4316:38
    at trackLeftHandSideExpressionAllowCall (/Users/nathan/Sites/git/sweet.js/lib/parser.js:4227:61)
    at parsePostfixExpression (/Users/nathan/Sites/git/sweet.js/lib/parser.js:2381:20)
    at /Users/nathan/Sites/git/sweet.js/lib/parser.js:4316:38
    at parseUnaryExpression (/Users/nathan/Sites/git/sweet.js/lib/parser.js:2430:16)

Looks like the operator is stripped there.

Edit: It only errors with multi-token names.

[natefaubion]

You can't overload operators (ie, both unary and binary):

binaryop ! 2 left { $l, $r } => #{ $l.send($r) }
var a = foo ! bar;
var b = !true;

SyntaxError: [macro] Macro `!` could not be matched with `true...`
2: binaryop ! 2 left { $l, $r } => #{ $l.send($r) }
            ^

The error reporting seems to be goofy, too. It's reporting the line of the macro definition instead of its usage.

[natefaubion]

Associativity seems a bit broken:

binaryop ($$$) 1 left { $l, $r } => #{ dollar($l, $r) }
binaryop (%%%) 1 right { $l, $r } => #{ modulo($l, $r) }

foo $$$ bar $$$ baz;
foo %%% bar %%% baz;

dollar(dollar(foo, bar), baz);
modulo(modulo(foo, bar), baz);

Shouldn't it be modulo(foo, modulo(bar, baz))?

Also a strange bug, if you leave off the semi colons and put the right-associative line first, you get an error, but not the other way around:

binaryop ($$$) 1 left { $l, $r } => #{ dollar($l, $r) }
binaryop (%%%) 1 right { $l, $r } => #{ modulo($l, $r) }

foo %%% bar %%% baz
foo $$$ bar $$$ baz

Error: Line 2: Unexpected identifier
[... baz ) ) dollar ( ( ...]
    at throwError (/Users/nathan/Sites/git/sweet.js/lib/parser.js:1887:21)
    at throwUnexpected (/Users/nathan/Sites/git/sweet.js/lib/parser.js:1923:13)
    at consumeSemicolon (/Users/nathan/Sites/git/sweet.js/lib/parser.js:1994:13)
    at parseStatement (/Users/nathan/Sites/git/sweet.js/lib/parser.js:3376:9)
    at /Users/nathan/Sites/git/sweet.js/lib/parser.js:4316:38
    at parseSourceElement (/Users/nathan/Sites/git/sweet.js/lib/parser.js:3790:20)
    at parseProgramElement (/Users/nathan/Sites/git/sweet.js/lib/parser.js:3805:16)
    at parseProgramElements (/Users/nathan/Sites/git/sweet.js/lib/parser.js:3833:29)
    at parseProgram (/Users/nathan/Sites/git/sweet.js/lib/parser.js:3869:16)
    at /Users/nathan/Sites/git/sweet.js/lib/parser.js:4316:38

[disnet]

Fixed the grouping and associativity problems for multi token operators. It had to do with the old bug of expanding inside of parens too many times. I seem to remember running into other issues related to that but can't find the issue where we discussed it before. Anyway it should be fixed now, and we might get a small performance boost too.

Still looking into the other issues you found.

[natefaubion]

I seem to remember running into other issues related to that but can't find the issue where we discussed it before.

We may have just discussed it in IRC. I've run into it a couple of times. If you never mutate source arrays, you'd be ok. But if your code is in a paren expr and you mutate the inner tokens, it would fail the second time around.

[disnet]

Ok, I think I've fixed all the issues you noticed. I've also got defining unary and binary versions of the same operator working. Let me know if you've run into anything else.

Also, how are you feeling about the current syntax?

[natefaubion]

After messing around with it, I think I'm definitely leaning towards a unified declaration of just operator. The macro definition is simpler, too, since you don't have to shadow the primitives:

macro op_assoc {
    rule { left }
    rule { right }
}

macro op_name {
    rule { ($name ...) }
    rule { $name } => { ($name) }
}

let operator = macro {
    rule {
        $name:op_name $prec:lit $assoc:op_assoc { $left:ident, $right:ident } => #{ $body ... }
    } => {
        binaryop $name $prec $assoc {
            macro {
                rule { ($left:expr) ($right:expr) } => { $body ... }
            }
        }
    }
    rule {
        $name:op_name $prec:lit { $op:ident } => #{ $body ... }
    } => {
        unaryop $name $prec {
            macro {
                rule { $param:expr } => { $body ... }
            }
        }
    }
}

I tried adding a rule for a JS body rather than a template, but I get a hygiene error with #{} saying name_stx isn't defined. Maybe that's just because I'm writing this in a test file and binaryop and unaryop are still macros, so it goes through more than one level of expansion and breaks. Works fine when I use the primitive form.