RFC for convenient, explicit closure capture using move($expr) expressions

[nikomatsakis]

Add move($expr) syntax inside closures, async blocks, and generators. A move($expr) evaluates the expression at closure-creation time and captures the result by value. This gives precise control over what a closure captures and when, without needing temporary variables outside the closure.

A prototype implementation is available thanks to @TaKO8Ki

Desired feedback:

• Choice of syntax

Important

Since RFCs involve many conversations at once that can be difficult to follow, please use review comment threads on the text changes instead of direct comments on the RFC.

If you don't have a particular section of the RFC to comment on, you can click on the "Comment on this file" button on the top-right corner of the diff, to the right of the "Viewed" checkbox. This will create a separate thread even if others have commented on the file too.

Rendered

[nikomatsakis]

This has been the subject of much prior discussion. I'm nominating for brief lang-team discussion in our upcoming Wednesday meeting.

@rustbot label +I-lang-nominated

[fmease]

Nightly rustc's parser also recognizes move( as the start of a move expression in all other places where an expression is expected, not just inside a closure body meaning it's a normal expression which I agree with. const _: () = move(0); is syntactically valid in the reference implementation, justifiably so.

Or are you saying that move($expr) should not be an Expression but rather an ExpressionIncludingMove and grammar rule ClosureExpression is to be updated to … ( ExpressionIncludingMove | -> TypeNoBounds BlockExpressionIncludingMove )*? I doubt that as it would be quite unconventional and inconsistent.

[nikomatsakis]

I agree, this section is too hand-wavy. I'll go make it more precise and compare against the implementation.

[fmease]

and the parser does not produce a move expression in other positions

As I just wrote, that would be quite surprising and unorthodox.

at the statement level, move(x) || y could parse as either a move expression followed by a logical-OR, or as part of a closure [emphasis mine]

I find the highlighted part a bit too vague; after all move( cannot possibly be a closure modifier unless you'd like to reserve the syntax for explicit capture list which I don't think is the case as you only seem to propose move[…] |…| … in Future possibilities.

I guess you meant to refer to the ambiguity where move(x) || y in statement expression context could be interpreted as two statements, move expression move(x) followed by closure || y? That's fixed by considering move expressions to be ExpressionWithoutBlock.

View changes since the review

[fmease]

Within a closure body, move(x) || y is parsed as a move expression (move(x)) followed by || y (which begins a nested closure literal).

Could you clarify that? The rustc implementation actually parses both || { move(x) || y } and || move(x) || y ("move(x) || y within a closure body") as a closure containing a logical OR which I agree with; there's no "nested closure literal".

If you actually want to parse || { move(x) || y } as a closure containing a move expression followed by a closure, then move expressions need to be an ExpressionWithBlock. However, that would imply that || { move(a) move(b) move(c) } would be syntactically valid (i.e., semicolon not needed to terminate expression statement move($expr)) which I don't think is intended?

[kennytm]

Yeah I find the last sentence very confusing. In the current spelling move(x) || y only makes sense as a logical-OR expression. Unless this sentence was written when it was spelled as move { x } || y?

[kennytm]

Suggested change

	let output_tx = move(output_tx);
	let mut output_tx = move(output_tx);

This is used in line 96 as &mut output_tx so output_tx itself must be mut.

This raises a question, what happens in the case we don't list the captures separately?

|| process(&move(input.vec), &mut move(output_tx), move(&cx.data));

I suppose this would error unless output_tx in the parent scope is also declared as mut?

And similarly, if the content of move is an rvalue (i.e. not a place expression), taking &mut should be always allowed?

|| f(&mut move({ output_tx }))

View changes since the review

[nikomatsakis]

Desired feedback: choice of syntax

View changes since this review

[nikomatsakis]

Desired feedback: choice of syntax

I'm not convinced that move($expr) is the right syntax. Here are some alternatives keywords I've considered:

• capture($expr) (new keyword, though)
• super($expr) (heavily overloaded keyword, I generally prefer move as it is already associated with closure capture rules, but open to others' takes)

And then there is the question of () vs {} (thanks @juntyr for raising it earlier!)

• move($expr)
• move { $expr }

[nikomatsakis]

The primary goals I think are clarity of semantics -- I am not sure if move conveys that well or not. capture strikes me as the most clear, but the potential conflict with existing code may be a major issue.

I suppose that capture { $expr } could be done as a "contextual keyword" and is unlikely to conflict with existing structs. I generally dislike contextual keywords.

[steffahn]

IMO, move(x) is bad because it’s too similar to C++’s std::move(x) but it does something entirely different.

Also regarding move { $expr }, I think that

move {
    block_contents()
    …
}

might look too similar to

move || {
    block_contents()
    …
}

---

I personally would like a new keyword like capture. I’m not a big fan of recycling keywords for entirely different purposes, anyway. move for closures was to indicate that implicit closure captures should all be done by moving. This new thing is for explicit closure captures (and fully flexible w.r.t. whether things are captured by moving, borrowing, or other preprocessing steps).

I’m unsure about style of parentheses. Both capture($expr) and capture { $expr } seem reasonable. The latter is perhaps syntactically more noticeable without syntax highlighting. The only downside I could see is that “capture” is a bit long.

[kennytm]

I suppose that capture { $expr } could be done as a "contextual keyword" and is unlikely to conflict with existing structs.

Haven't we learnt from try { ... } 😅? As an expression capture cannot just be a contextual keyword.

And yes struct capture with a small c does exist.

[steffahn]

move($expr) is then replaced with a reference to this temporary

why a reference? Can’t you use the captured thing by-value? Or… wait a minute… do you not actually mean a reference? [I’m only noticing this alternative intepretation on second read.]

To improve this confusion:

when the closure is called, move($expr) functions as a place expression referring to this temporary

Also, perhaps spelling “temporary” longer as “temporary variable” is easier to read for more people?

The expression $expr will execute when the closure is created and then moved into a temporary variable captured by the closure; when the closure is called, move($expr) functions as a place expression referring to this temporary variable.

The use of “moved to” in this sentence is perhaps a fun trick/mnemonic if the keyword is still move, but we could just as well write “stored in” or “assigned to” or something like that.

Here’s a possible rewording, though less comprehensive edits are of course possible.

Suggested change

We propose to introduce a `move($expr)` expressions that can appear in a closure or a future. The expression `$expr` will execute when the closure is created and then moved into a temporary captured by the closure; `move($expr)` is then replaced with a reference to this temporary. This can be used to capture a clone of a value (`move(vec.clone())`) but also other derived values (e.g., `move(vec.len())`).

We propose to introduce a `move($expr)` expressions that can appear in a closure or a future. The expression `$expr` is evaluated when the closure is created and its value is stored in a temporary variable captured by the closure. When the closure is called, `move($expr)` functions as a place expression referring to this temporary variable. This can be used to capture a clone of a value (`move(vec.clone())`) but also other derived values (e.g., `move(vec.len())`).

View changes since the review

[steffahn]

I think, if closure syntax is viewed as a form of quasiquotation, then these move(…) expressions basically just antiquotation, right? I’m not sure what the best concrete prior art to cite on this would be but I believe the comparison is quite useful.

View changes since the review

[steffahn]

IMO, move expressions don’t solve the same problem fully. If a variable us used multiple times in the closure, then move expressions don’t really allow any significant shortening of boilerplate.

Current Rust (variant 1)

let foobar_ = foobar.clone();
use_closure(move || {
    foobar_.method1();
    foobar_.method2();
});

foobar.still_usable_here();

(variant 2)

use_closure({
    let foobar = foobar.clone();
    move || {
        foobar.method1();
        foobar.method2();
    }
});

foobar.still_usable_here();

With explicit capture (with shorthand supporting a method like .clone()):

use_closure(move(foobar.clone()) || {
    foobar.method1();
    foobar.method2();
});

foobar.still_usable_here();

With move($expr):

use_closure(|| {
    let foobar = move(foobar.clone());
    foobar.method1();
    foobar.method2();
});

foobar.still_usable_here();

---

I guess what’s really missing is the potential for any shorthand to avoid the need for listing foobar twice? The point where you need to type every cloned variable name two extra times is where things get annoyingly verbose.

That’s especially if you introduce a second usage, e.g. going from

|| {
    move(foobar.clone()).method1();
}

to

|| {
    let foobar = move(foobar.clone());
    foobar.method1();
    foobar.method2();
}

involved introducing three mentions of foobar.

---

One thing that move($expr) does however allow is to shorten things with a simple macro. (Maybe that’s all we really need?)

use_closure(|| {
    capture_clone!(foobar);
    foobar.method1();
    foobar.method2();
});

foobar.still_usable_here();

use_closure(|| {
    // expands to a list of `let $ident = move($ident);` statements
    capture_clone!(foobar, baz, frobnication);

    foobar.method1(baz);
    baz.call(frobincation.wow());
    foobar.method2(baz, baz);
});

foobar.still_usable_here();

View changes since the review

[ds84182]

Perhaps I'm an outlier but many expect that, in the absence of control flow, programs are executed in the order declared within the source file. This is a departure from that, as these expressions are not evaluated in the same place as its neighbors.

Expression evaluation escapes execution.

One can argue that constants do the same thing, but constants have no side effects so the site of evaluation cannot be observed (execution duration is not a side effect; this should not be debated here, panics neither as it does not produce a program to have side effects in).

If this should exist, the expressions must never diverge (as in panic, infinite loop, abort). At that point the side effects of this early evaluation are miniscule, and therefore this would not cause much problems. It would require a new "color" of function for non divergence, which is probably years away at this point. (In other words, it's not const and not pure but a secret third thing 🙂)

But in its current state I think it will be fairly easy to encounter new footguns unique to Rust. This is solving a common problem, meaning it'll be commonly used, and at times accidentally or purposefully misused. A panic on closure creation is unprecedented, and suddenly macros containing closures have to deal with closure creation possibly executing arbitrary code.

Now, if I understand correctly, the suggested use idea allows the clone to be syntactically elided because the Use trait should only be implemented on types with "cheap clones" (which should not diverge except in extreme circumstances). Therefore this matches the expected behavior of a program as the inserted clone is negligible.

View changes since the review

[Nadrieril]

in the absence of control flow, programs are executed in the order declared within the source file

I'd argue that's still the case/not less the case! The code inside a closure was never going to be executed right away, it's just that now building the closure may also be executing a few extra operations, in the order in which the move expressions are. Not saying it's not surprising but I'd say: before, a closure was one big "hole" in the order of the execution; now it's a hole-with-holes or like a kind of sieve.

[dlight]

This feedback is puzzling, because .use isn't implicit and doesn't hide anything. It removes emphasis on cloning, just like ? removes emphasis on the ceremony of forwarding an error to the caller. Sometimes de-emphasizing the cost of a clone is wanted and helps with legibility. But like in the ? case, there is still a syntactical marker that points to what is happening.

In any way, is .use and move() compatible with each other? Can we imagine a future where we get both?

I'm asking because .use isn't just about closure captures, and it made clone-heavy code feel easier to read.

View changes since the review