Either make assignments a statement type or allow 'my' declarations in expressions

[masak]

Just a thing that keeps bothering me. A very subtle inconsistency.

In Perl 5 and Perl 6, assignment is an operator, though admittedly a fairly weird one (due to the lvalue semantics). In Python, assignment is a statement type.

say(a = 5);    # allowed in Perl 5/6 and 007, illegal syntax in Python

007 sides with the Perls on this one, and that in itself has never bothered me much.

But 007 ends up siding with Python on a related feature, and that's where the inconsistency is: my declarations in 007 are a statement type, unlike in Perl 5 and Perl 6:

say(my a = 5);    # allowed in Perl 5/6, illegal syntax in Python and 007

Now, things are not identical syntactically in Python, which makes this inconsistency a little harder to spot: Python does not have a variable declaration keyword as such. (And I think that's a mistake, but that's another story.) Instead, the first assignment to a name in a given scope sort of also counts as a declaration. The actual syntax in Python for the above would be print(a = 5), except that here the a = means that we're trying to pass a (nonexistent) keyword argument to print.

Anyway. Maybe we should go one way or the other. I haven't decided which.

• Maybe we should turn my declarations from a statement type to an expression type. (More a term than a prefix, I think.)
• Or maybe we should turn assignments from an operator to a statement type.

So far I can only list the advantages of either, as far as I can see.

Advantage of turning my into an expression

It would allow the following idiom:

while (my c = getChar()) != EOF {
    # ...
}

Which we currently have to write like this:

my c;
while (c = getChar()) != EOF {
    # ...
}

It's a small thing, but I'm generally in favor of colocating declarators with the first use of a variable. Having to declare it on top of the loop without initializing it... feels like a small inconvenience.

(I don't think the scope of c should change just because we declare it in the while condition, though. A contrived case could perhaps be made that it should... but I say it's outside the braces of the loop (and it's not a parameter), so it makes sense that it still be visible after the loop block too.)

Disadvantage of turning my into an expression

We wouldn't be able to do #199.

Advantage of turning assignment into a statement type

Simplicity would be the main advantage here. It's more consistent with how my already works in the language.

Note that we can still (like Python) allow chained assignments a = b = 42;. That's just part of the new statement syntax. I think we'd say no to my a = my b = 42; on grounds of sanity. (But my a = b = 42; would presumably be OK.)

Disadvantage of turning assignment into a statement type

I'm struggling to even express the while loop idiom when assignment is a statement type. Something like this, I guess:

my c = getChar();
while c != EOF {
    # ...
    c = getChar();
}

You may or may not prefer the duplication to the increased complexity of assignment expressions.

(And Perl 5 has a continue block after loops to handle the case where we have early next statements inside the loop, but want to do the same kind of "increment" logic between iterations.)

[masak]

Adding another one to the "Advantage of turning assignment into a statement type": two other issues (#163 and #237) have identified "assignment precedence" as something that needs to be special-cased. But if assignment weren't actually part of expressions at all, but something outside of it, that concern wouldn't exist in the first place.

[masak]

Adding a dubious one to the "Disadvantages of turning my into an expression" pile: assuming #257 happens (which I think is more likely than not at this point), my declarations might be yet another thing we might want to annotate. Doing so on an assignment statement feels a whole lot more straightforward than doing so inside of an expression:

@Immutable
my pi = 314;    # straightforward

say(@Immutable my pi = 314);    # hmmmm

Though I can certainly argue this one both ways. It looks quite reminiscent of annotated parameters, for example.

Also, I'm not 100% convinced we even want annotations on my declarations, so it's not a super-strong argument either way.

[masak]

I'm more and more leaning to making assignments a statement type. Assignments are weird. As an operator they're easily a macro, or maybe even "I give up, let's just make this a magical primitive".

By the way, it's interesting to think how the landscape in #214 changes if we change assignments to be a statement type.

[masak]

I just pushed up the branch "assignment-statement" — still a work in progress, but what it does is just exploratorily replace Q::Infix::Assignment with Q::Statement::Assign.

There seems to be a parser ambiguity somewhere. The new parsing rule looks like this:

rule statement:assign {
    <termish>
    '='
    <EXPR>
}

(<termish> is not super-exactly right here; I want the postfixes but not the prefixes. On the other hand, I can't quite motivate why I never want the prefixes, except by saying that the built-in ones we have always rvalue-ify lvalues. But that's not set in stone, I guess; someone might define a prefix that produces lvalues. So <termish> is maybe exactly what we want.)

Anyway, I didn't get this to work for something as simple as x = 2;. I think I either have some mundane whitespace-related issue (solvable by writing more tests) or there's a parsing conflict between the new statement:assign and EXPR.

Need to investigate further.

[masak]

Well! This is timely: PEP 572.

[masak]

I want to make progress on this one, so let's tally things up so far.

(I also have a personal favorite, but I'd like to see the same side fall out "objectively".)

Advantages of making assignment a statement

• It's closer to Python (but this argument gets weakened by the presence of PEP 572)
• It'd still allow Implement multiple comma-separated variable declarations in 'my' statement #199
• It'd move a kinda-weird operator to be a special form

Advantages of making my a term

• It's closer to Perl 6
• There are genuine use cases that are better expressed this way (notably, declaring things en-passant in loop heads)
• While we don't have other "weird" operators (that treat something as an lvalue) in core, we're expecting to have them as macros (e.g. postfix:<++>)

That's three against three, except I consider the first two points in favor of the assignment statement to be somewhat weakened. This pretty much sums up my current personal stance, too: I'm slightly in favor of removing Q::Statement::My and just throwing my in as a special-cased term.

Probably going to try that out in a branch, too.

I noticed I called out #163 and #237 above as arguments in favor of making assignment a statement, but... I don't know. I think those are too early to tell. I don't immediately have a sense that making assignment a statement will help there. Maybe a solution for those two is to have a way for the parser to start parsing at a given precedence level (like Rakudo does)?

[masak]

Another advantage of making my a term: It's more orthogonal.

When I first implemented Q::Statement::My, it had a small expression as its second field; my x = 42; would store x in the identifier field and x = 42 in the expression field. It captures well the split between compile-time declaration and runtime effect, but the x needed to be stored twice, and there is much too much leniency with the expression; anything could be stored in there, and that's not really what we want to express.

Then, inspired by IntelliJ's Java PSI, I managed a simplification in which only the thing assigned is stored, not an expression containing the assignment. Obvious in retrospect. (Note also how neatly the semipredicate problem is solved in this case. The expression always contains either a Qtree for an expression, or None if there's no expression.)

Making my a term would bring a further simplification; now we don't need to bother with assignment at all — that is, assignment becomes completely orthogonal to my, since the my x just has a declarational effect at compile time, and then behaves like (the lvalue) x at runtime.

In my view, that's so neat and explainable that it counts as an advantage on its own.

[masak]

#160, reluctant as it is, would also count as a vote in favor of my as a term.

[masak]

According to this text (page 70:10; strange numbering scheme), C++98 introduced being able to declare variables in if statements, and that was a new idea at the time. It did this not by changing declarations into a form of expression, though, but by allowing the condition of an if statement to be a declaration. while loop conditions can also be declarations.