super-let/

[utterances-bot]

Rust temporary lifetimes and "super let" - Mara's Blog

The lifetime of temporaries in Rust is a complicated but often ignored topic. In simple cases, Rust keeps temporaries around for exactly long enough, such that we don’t have to think about them. However, there are plenty of cases were we might not get exactly what we want, right away.
In this post, we (re)discover the rules for the lifetime of temporaries, go over a few use cases for temporary lifetime extension, and explore a new language idea, super let, to give us more control.

https://blog.m-ou.se/super-let/

[dvdsk]

I like the idea, especially combined with limiting temporary lifetime extension in the future. Only the name feels confusing, I guess in my mind super is just too tightly linked to modules.

What about extend let? That also makes it easier to remember that the syntax has to do extending lifetimes.

let writer = {
    println!("opening file...");
    let filename = "hello.txt";
    extend let file = File::create(filename).unwrap();
    Writer::new(&file)
};

[m-ou-se]

I don't actually care much about the syntax! I just picked super for now because it's already a keyword today, meaning that this wouldn't need a new edition, and "super let" sounds kinda catchy. It's useful to have a simple name when talking about a new idea. ^^

[jdonszelmann]

I love the idea! I've definitely had this plenty of times. I like that you raise the issue of diagnostics. Those are super important to me, especially the warning when to remove super let. I'm a bit afraid that otherwise super let will just become a default thing to try when lifetimes don't work out: just try to insert some supers before some lets. I've seen many people just try random shit if it doesn't work, without understanding the underlying problem. If everything then becomes a super let, some subtle bugs might start occurring with mutex guards for example. I am definitely kind of in love with placing functions though, even if it's not fully worked out. I've wanted something like that for a long time.

[lebensterben]

what about let*?

[illicitonion]

Thanks for the really clear write-up, I'm super excited by this, and really like the framing as super let.

I wrote a blog post about a subset of these use-cases a while ago, and put together a hacky proof of concept proc macro to emulate it at function scope (rather than arbitrary scopes) and some possible alternative syntaxes (my favourite of which was let hoisted which is very close to super let).

I think super let would be a really nice way of solving these issues, and would love the place-temporaries-in-caller extension!

[Lamby777]

That part about the Some(&mut file) was so relatable, I used to be so confused working with strings when I was new because of if statements being their own block. This sounds nice, I just hope it won't end up causing less readable code in practice.

[Nadrieril]

Oh wow I am loving the idea of function-level super let. Problem is, once you have that, you'll need syntax so that a placing function can call another placing function to its caller, at which point you've essentially got an effect system. Only missing is syntax to "catch the effect", i.e. to call a placing function while specifying yourself where the value is placed, and boom you've solved placement new (kinda (maybe)).

[SpriteOvO]

I'm a bit concerned that the super let will make code hard to read and understand. Before super let, a reference is just a reference, it means pointing to a exact value somewhere. With super let added, a reference is not just a reference anymore, it is probably like... owned a value. I mean, it's kind of weird that a reference owned a value.

[max-heller]

@lebensterben let* is used in some other languages (e.g. Racket) with very different semantics

[lebensterben]

@lebensterben let* is used in some other languages (e.g. Racket) with very different semantics

I am a lisp programmer and I am aware of it. IMO let* is easier to read than super let and * (star) is commonly used in many scientific literature to denote something similar but distinct to the un-starred one. An alternative is let'.

[rsalmei]

I liked the idea, but I think it would be clearer if the "super" mark was put on the statement that grabs the reference and triggers its extension, somehow...
Instead of:

    super let file = File::create(filename).unwrap();
    Writer::new(&file)

Something like:

    let file = File::create(filename).unwrap();
    Writer::new(super &file)

Because that's the line that needs a temporary lifetime extension.

[madsmtm]

As a macro author myself, I don't think the motivation of "writing macros more easily" really carries that much weight. Even in std, it only comes up in two places: pin! and format_args!, and both cases can be worked around with private implementation details (arguably, format_args! should work around it with private implementation so that we can have that functionality today, just like is done for pin! (though I recognize it may interfere with the other work to improve format_args!)).

That said, I think this feature is interesting, and the other use-cases are definitely important!

[ppershing]

I too think that feature is interesting. Current rules about lifetimes can be confusing and provide unclear behavior in cases such as mutexes/destructors running visible side-effects.

As for the syntax (e.g. the bikeshedding part) - I found super let too confusing on the first glance. My mind was like - "what does it do better than normal let"? But what about let(super) ... ? This wold be similar to how visibility works and would let you later on do the parent-function scoping as well, e.g. by let(caller) ...

[BartMassey]

Great article with a really well-written exposition of the issues! Thank you very much for the obvious work that went into it.

I think offhand — this is not a well-thought-out comment, I'm afraid — that the actual mistake was made l long ago. The problem that keeps coming up that makes "as long as reasonably possible" lifetime extension for everything undesirable seems to be automatic release of guard objects. (Specifically lock guards and borrow guards in examples, but the guard pattern is in a few other places as well, I think?)

Just because you can make explicit guard drops unnecessary doesn't meet you should. Indeed, I would prefer the opposite: some kind of lifetime checking to statically require that a guard object be explicitly released (somehow). This would work nicely with maximal lifetime extension while statically protecting programmers from the late-release footgun in a way the current solution does not.

With this plan the lifetime of your guard can be as long as the language knows how to make it. If the programmer doesn't explicitly release the guard, they get a compiler error they have to fix. Such an analysis will, like lifetime analysis itself, necessarily be imprecise — some claims of lack of explicit release by the compiler will be wrong. That's ok: we've shown programmers can live with that if it's reasonable.

I think what I'm asking for here is part of "linear types" somehow? I don't know. Like I said, not a well-thought-out-comment. In any case, Rust is a ship that has sailed: it's way too late to go try and undo this probably.

Watching less-experienced Rustaceans trying to deal with early drops is painful. Schemes like super let and Niko's .let that allow shortening lifetimes when not explicitly told not to seem to me to be compromises that step delicately around guard release in an attempt to get luckier (sort of) about correctness, while being less delicate with the ergonomics of programming. As a (really) old-school parallel person and a long-time programming instructor, I'm not sure this is the best tradeoff. The folks doing explicit parallelism are probably better able to defend themselves than the folks trying to figure out why their "obviously ok" code is rejected by the compiler. Maybe. I dunno. In any case, I've been telling my students for a while to just always explicitly drop() their guards, to avoid the kind of surprise being worried about here. If they forget — well, maybe they'll get lucky and the default guard release point will be good enough anyhow.

Anyway sorry for the long rambly post: hope it's ok. Again, thanks for a really thought-provoking and informative piece.