Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign upnon-lexical lifetimes #2094
Conversation
nikomatsakis
added
the
T-lang
label
Aug 2, 2017
This comment has been minimized.
This comment has been minimized.
|
(Sorry, reactions only let you do one, this RFC deserves many) |
This comment has been minimized.
This comment has been minimized.
ErichDonGubler
commented
Aug 2, 2017
|
Verdict: let's do it! |
This comment has been minimized.
This comment has been minimized.
nicoburns
commented
Aug 2, 2017
|
I have a question about a slightly modified version of an example in the RFC. Under the proposal in the RFC, would the following be valid? fn process_or_default() {
let mut map = ...;
let key = ...;
match map.get_mut(&key)
Some(value) => {
println!("{:?}", value);
map.insert(key, V::default());
},
None => {
map.insert(key, V::default());
}
}
}Note that unlike the example in the RFC, the |
This comment has been minimized.
This comment has been minimized.
|
If I understand correctly, in this case, lifetime of |
Gankro
referenced this pull request
Aug 3, 2017
Closed
RFC 2094: 👏 M O T H E R 👏 H U G G I N G 👏 N O N 👏 L E X I C A L 👏 L I F E T I M E S 👏 #32
This comment has been minimized.
This comment has been minimized.
|
Disclaimer: haven't read layers 2-5 yet, and have been skimming the end of layer 1 to try to understand the earlier parts. I probably need to understand a bunch of this stuff to update the nomicon, so understanding it well is pretty important to me. First off, when describing the algorithm for solving constraints, you introduce the notion of "exiting" the lifetime Second off, I'm having trouble rectifying these statements:
So But...
So Means I think my issue might be trying to think of the lifetimes without the |
This comment has been minimized.
This comment has been minimized.
RReverser
commented
Aug 3, 2017
|
Perhaps I missed it from a quick glance, but are there mechanics that will allow opt-out from non-lexical lifetimes to plain old blocks for FFI interactions? Currently one can expect that values in the block will be deallocated in the end of the block, so it's "safe" to pass a pointer to such values to FFI code that needs to interact with these pointers in other FFI calls till the end of the block, even though Rust can't see or know whether the first FFI call just used a reference/pointer temporarily or stored somewhere else. (This is a bit vague description, I can provide an example a bit later when not from the phone, if it will be needed) |
This comment has been minimized.
This comment has been minimized.
|
An example I feel would be interesting, because I'm not exactly sure what the issue (but considering it's |
This comment has been minimized.
This comment has been minimized.
|
Not only what @xfix said, but lifetime inference cannot affect code generation. |
This comment has been minimized.
This comment has been minimized.
RReverser
commented
Aug 3, 2017
•
Ah okay, that was the confusing bit - for some reason, I thought that non-lexical lifetimes would also bring earlier destruction for objects that otherwise would live till the end of the block. |
This comment has been minimized.
This comment has been minimized.
burdges
commented
Aug 3, 2017
|
If I understand, this code would now compile where currently the 2nd
It seemingly follows that non-lexical lifetimes limit the ability of wrapper types to impose temporary restrictions on data structures you apply them to, meaning this is not only a concern for FFIs. It'd be easy to fix this with some attribute, say |
This comment has been minimized.
This comment has been minimized.
|
@burdges A drop counts as an use so |
aturon
referenced this pull request
Aug 3, 2017
Open
Language ergonomic/learnability improvements #17
This comment has been minimized.
This comment has been minimized.
|
Thank you very much for discussing how error messages will be improved by this proposal! I believe that the new errors, with the borrow / invalidation / usage sites clearly highlighted, will go a long way to helping people understand what is wrong and how to fix them.
I fully agree that the usage of the term "lifetime" is overloaded, and in a clear restart we would want to avoid using it. This feels like one of the trickiest aspects of Rust for people coming from "closely related" languages where they have to think about lifetimes. When people "get" that a lifetime is actually "the time period in which the address of a variable remains constant", that's when a lot of other things start to click for them.
That said, I disagree with this aspect. This sounds like we are trying to stop using the term lifetime. I worry that such a direction will make things worse. In such a world, we'd have existing literature (blog posts, the book, Stack Overflow, etc.) that uses the term "lifetime" but the error doesn't. There'd be no obvious way for a new user to connect the two.
This is much more palatable to me. |
aturon
assigned
nikomatsakis
Aug 3, 2017
This comment has been minimized.
This comment has been minimized.
Florob
commented
Aug 4, 2017
|
I'm still a bit concerned about NLL, because to me it actually makes reasoning about references a lot harder. Specifically where a reference can be used without invalidating the program is no longer straight forward to answer. In particular the fact that introducing a use of a reference can invalidate a previously valid use of a referent feels like unfortunate error-at-a-distance behavior. That said, the benefits probably still outweigh this concern by far. |
This comment has been minimized.
This comment has been minimized.
vorner
commented
Aug 4, 2017
|
Hello I like it, these annoying things bite me from time to time (I guess it's less often now, because I avoid the pitfalls unconsciously already). I'm not an expert in the area, and I didn't follow every argument to the ground, but I think there are two small places that could be extended a bit:
Eg, this'll compile: struct X<'a> {
val: &'a usize,
}
{
let x = 42;
let y = X { val: &x };
x = 50;
}However, it'll stop compiling if a drop implementation is added to |
cramertj
referenced this pull request
Aug 7, 2017
Open
Better temporary lifetimes (tracking issue for RFC 66) #15023
This comment has been minimized.
This comment has been minimized.
fn main() -> ! {
let x: usize;
let y: &'static x = &x;
loop { }
}Actually, makes perfect sense to me based on the CFG-node contextuality of of the subtyping rules. Furthermore, this would be potentially very useful for embedded programming CC @japaric. Yes it is unsound in the presence of concurrency right now, but I'd love to delve into trying to make it work. IMO just automatically throwing it out would be like just throwing out |
This comment has been minimized.
This comment has been minimized.
Is it? |
This comment has been minimized.
This comment has been minimized.
|
@RalfJung well the scoped threads example below---which I missed at first :). But actually maybe not; I dislike the premise of "the guard must be dead because its destructor is never run"---I think we should have a |
This comment has been minimized.
This comment has been minimized.
|
This thing doesn't terminate, so I don't see a conflict with scoped threads. In fact I think I would prove this function safe in the RustBelt formal model. EDIT: I assume you mean this? Not sure what the problem is with that example; there's too much left implicit for me to follow @nikomatsakis here. Where is 'static even coming up? |
This comment has been minimized.
This comment has been minimized.
|
I honestly wouldn't mind myself if this example would work, however there is an issue with unwinds which can be used as alternative means of escaping a function - not in this particular example, of course, but in practical examples of this. Even a simple integer addition can unwind in Rust, which is problematic for examples more complex than just This seems safe without unwind, but having borrowck depend on whether panic is unwind or not doesn't sound like a good idea - whether code would compile would depend on global compilation options. |
This comment has been minimized.
This comment has been minimized.
|
I think we should take this to a separate issue---I do really like that the RFC-repo idea to avoid a mega thread. @xfix Well, I've spilled much ink on features that benefit from no-unwind-specific borrow checking https://internals.rust-lang.org/t/a-stateful-mir-for-rust/3596 so it wouldn't be the first time I crossed that bridge :D. |
This comment has been minimized.
This comment has been minimized.
|
The infinite loop example here: let mut foo = 22;
unsafe {
// dtor joins the thread
let _guard = thread::spawn(|foo| {
loop {
// in other thread: continuously read from `foo`
println!("{}", foo);
}
);
// we assume `_guard` is not live here, because why not?
loop {
foo += 1;
}
// unreachable drop of `_guard` joins the thread
}Is unsound because both the main and the spawned thread can access However, that API is unsafe anyway, because of let scope = Scope::new();
let mut foo = 22;
unsafe {
// dtor joins the thread
mem::forget(thread::spawn(|foo| {
loop {
// in other thread: continuously read from `foo`
println!("{}", *foo);
}
));
// there's no guard, the gates are open, the threads are spinning
loop {
foo += 1;
}
} |
This comment has been minimized.
This comment has been minimized.
|
@arielb1 nikomatsakis/nll-rfc#35 I forgot to link it back here. |
withoutboats
added
the
Ergonomics Initiative
label
Aug 14, 2017
This comment has been minimized.
This comment has been minimized.
Nashenas88
commented
Aug 23, 2017
•
|
@Florob, what if there was a way to actively visualize the lifetime while you were editing? My original visualizer idea is somewhat invalidated by this RFC, but this seems like only a small deviation from what I was working on. Imagine the following workflow:
I imagine such a model would make it easier (not necessarily easy) for someone to learn to understand the new system. @vorner, you reminded me of an idea I've had a few times. What if we showed the user the drop statements that get injected, and point to those when explaining the issues with outlives (doesn't help with named lifetimes). Quite a few times I've gotten confused when the error description seems to point to the exact same spot, the final closing brace of the block. E.g.: struct X<'a> {
val: &'a usize,
}
{
let x = 42;
let y = X { val: &x };
// ^~ borrowed here, etc.
x = 50;
// ^~ modified here, etc.
// Some output to distinguish that this is compiler generated and not user input
drop(y);
// ^ but must live until here, etc.
// Some output to distinguish that this is compiler generated and not user input
} |
This comment has been minimized.
This comment has been minimized.
rfcbot
commented
Sep 6, 2017
•
|
Team member @nikomatsakis has proposed to merge this. The next step is review by the rest of the tagged teams: No concerns currently listed. Once these reviewers reach consensus, this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See this document for info about what commands tagged team members can give me. |
rfcbot
added
the
proposed-final-comment-period
label
Sep 6, 2017
This comment has been minimized.
This comment has been minimized.
alexanderjsummers
commented
Sep 8, 2017
|
I've been reading the proposal (which generally sounds like an excellent thing for the language, and is mostly very clear). After some discussions with @vakaras here, I have some smaller suggestions about various parts the document (I'll make PRs/issues) but the main queries we have are about Layer 5 (how this fits into the borrow checker). In particular, the last of the three rules for the "transfer function" doesn't seem to be correct, based on our understanding so far. This rule states that:
For example, considering the following code: struct Cell {
value: u32,
}
fn f(x : Cell, y : Cell)
let c: &mut Cell = &x; // (0) borrow on x
let v = &mut (*c).value; // (1) borrow on x's field
c = &y; // (2) reassign c to borrow on y
use(x); // (3) not allowed; v is still in scope
use(v);
use(c);I expect (and get, from the prototype) an error telling me that using |
This comment has been minimized.
This comment has been minimized.
|
First, some nits. I am adjusting the example to get the struct Cell {
value: u32,
}
fn f(x : Cell, y : Cell)
let c: &mut Cell = &mut x; // (0) borrow on x
let v = &mut (*c).value; // (1) borrow on x's field
c = &mut y; // (2) reassign c to borrow on y
use(x); // (3) not allowed; v is still in scope
use(v);
use(c);
}Now, onto your question:
I'm not sure if the bug is in the text or your interpretation of it, but let me explain what I expect to happen and we can decide. =) First off, the assignment Note also that region inference comes as a first phase, before this rule is applied. This means that the lifetime of (Something about this setup bugs me a bit; I have wondered if it would make sense to integration the borrowck with the region inference step. But for now, that's not how it is specified.) |
This comment has been minimized.
This comment has been minimized.
|
One belated bike-shed on terminology: We've used "non-lexical" for a while now, but I worry that people will take it as a synonym for "dynamic", influenced by "lexical vs dynamic scope". I think it's important for learning to understand that this a still a static system, in that a given CFG node is always in the same unchanging, known set of lifetimes. That staticness in my view makes both learning NLL and debugging code with it far far easier. |
This comment has been minimized.
This comment has been minimized.
|
@arielb1 also raised the question of finding another name for this proposal, though I tend to think that the current name is "ok", since this isn't a feature that people will "interact with" for the most part (i.e., once its in, it'll just be "lifetimes" -- a term with which I have some issues anyhow, but that's another story!). |
This comment has been minimized.
This comment has been minimized.
alexanderjsummers
commented
Sep 8, 2017
|
Thanks a lot for the speedy clarifications. As you might have guessed, the example had gone through a little by-hand massaging, which obviously wasn't totally error-proof.. I've copied our original file at the end of this post. I think we mixed up the lvalue and rvalue for the first loan, which is why we had the idea that it gets cancelled too. Now it seems clear how we can get the error we do, with respect to the given rules. What I find a bit counter-intuitive is that the notion of loans considered in scope (with respect to the discussion in the RFC) doesn't correspond with the notion of borrows which are still active, if I understand correctly. For example, at point (2), as you've explained, the second loan gets cleared. But the borrowed reference Here's the original code and interaction with the prototype. By the way, is the information as to which loans get cancelled reported in some way? The regions reported seem to correspond to the scope of the actual borrowed references, and not the loans. struct List<+> {
value: 0,
}
let x: List<()>;
let y: List<()>;
let list: &'list mut List<()>;
let v: &'v mut ();
block START {
x = use();
y = use();
list = &'b1 mut x;
v = &'b2 mut (*list).value;
list = &'b3 mut y;
use(x);
use(v);
use(list);
}
// With use(x):
// Message: ../test/example.nll: point START/5 cannot read Var(Variable { name: "x" }) because Var(Variable { name: "x" }) is mutably borrowed (at point `START/2`)
// Loan {
// point: START/2,
// path: Var(Variable { name: "x" }),
// kind: Mut,
// region: {START/3, START/4, START/5, START/6, START/7} }
// Loan {
// point: START/3,
// path: Extension(Extension(Var(Variable { name: "list" }), FieldName { name: "*" }), FieldName { name: "value" }),
// kind: Mut,
// region: {START/4, START/5, START/6} }
// Loan {
// point: START/4,
// path: Var(Variable { name: "y" }),
// kind: Mut,
// region: {START/5, START/6, START/7} }
// If we re-run with use(x) commented out (which shifts some line numbers):
// Loan {
// point: START/2,
// path: Var(Variable { name: "x" }),
// kind: Mut,
// region: {START/3, START/4, START/5, START/6} }
// Loan {
// point: START/3,
// path: Extension(Extension(Var(Variable { name: "list" }), FieldName { name: "*" }), FieldName { name: "value" }),
// kind: Mut,
// region: {START/4, START/5} }
// Loan {
// point: START/4,
// path: Var(Variable { name: "y" }),
// kind: Mut,
// region: {START/5, START/6} }
// assert 'list == {START/3, START/4, START/5, START/6};
// assert 'v == {START/4, START/5};
// assert 'b1 == {START/3, START/4, START/5, START/6};
// assert 'b2 == {START/4, START/5};
// assert 'b3 == {START/5, START/6};Also, as a (possibly-unrelated) side-question, I'm not sure why the region for the first loan contains |
This comment has been minimized.
This comment has been minimized.
What gets cancelled on an assignment is not loans that the overwritten variable refer to, its loans that refer to the overwritten value (and are now "orphaned"). e.g. in your code: struct List<+> {
value: 0,
}
let x: List<()>;
let y: List<()>;
let list: &'list mut List<()>;
let v: &'v mut ();
block START {
x = use();
y = use();
list = &'b1 mut x;
// This creates a lifetime constraint where `list` (and therefore the borrow
// of `x`) must live as long as `v` is.
v = &'b2 mut (*list).value;
// (*list).value is now borrowed (and "dynamically" refers to x.value), so e.g.
// use(&list); //~ ERROR cannot use `list` while `(*list).value` is borrowed
list = &'b3 mut y;
// (*list).value now refers to a completely different thing (y.value), and
// is therefore no longer borrowed. The old borrow for `(*list).value` is
// now "orphaned" - it is not accessible by any path derived from the old
// path. The lifetime constraint still remains.
// use(x); // ignore this, I don't want an error
use(&list); // non-consuming use, in any order
use(v);
use(list);
}
|
This comment has been minimized.
This comment has been minimized.
Indeed, I do understand what you meant, and I too find this interaction somewhat surprising, though it seems to by and large "do the trick" (i.e., it mostly accepts the programs I want it to, and seems valid). I feel like there may be a more elegant way to express it. This is what I was trying to refer to when I wrote:
As @arielb1 points out, though, what's really happening here is that we are eliminated a kind of "false sharing" (to abuse a term). That is, we have on record a loan of One could imagine instead using a kind of SSA-like-rewriting to achieve a similar effect, at least for simple cases like local variables; but it'd be more work to do the same for paths like Anyway, I'd be not be opposed to exploring such alternate formulations, though I think we need not block on it. To my mind, what's being RFC'd here are the high-level concepts: the details of the checks we perform will be tweaked over time no doubt. (Though I would like the RFC and prototype to serve as the basis for a reference document and implementation that we can keep up to date.) |
This comment has been minimized.
This comment has been minimized.
alexanderjsummers
commented
Sep 11, 2017
|
Thanks a lot for the further details and clarifications. I'm no longer worried that there's a technical issue here - the rules indeed seem to do the "right thing". If the idea is to evolve this document into documentation later, my suggestion would be to pick a different name from "loan" if possible, for what is tracked (but maybe this terminology is fixed already). My conceptual understanding of borrowed references has always been that the references take some capabilities from each other, and that these are for a particular location, and a particular lifetime. I guess the conversion to SSA would make the distinction between the borrows and loans more explicit, since the rules for cancelling loans would be tied to distinct syntactic paths in the program, but I guess the distinction (which was what I was missing in my first comment) would still be the same in the end. Maybe converting to SSA for the sake of an example could be a(n alternative) way of explaining how the borrow checker rules can be thought about in terms of why they make sense, but I guess it would just be for illustration if so, and as you say, for general paths it might not be all that pretty. @nikomatsakis I also find it interesting to think about whether these phases could somehow feedback to each other meaningfully - it doesn't yet seem clear to me why this kind of phased checking naturally arises, but it indeed seems to achieve the right results, too! I might ponder this a bit and ping you an email if I think I have any sensible thoughts. I have a few smaller suggestions about the document, in terms of things I would find helpful to clarify. Is it best to comment here, make pull requests (but in some cases I don't actually know what I would propose yet), or add issues etc.? Or indeed, keep them to myself? :) |
This comment has been minimized.
This comment has been minimized.
PhilipDaniels
commented
Sep 11, 2017
|
If you are still looking for a name for the concept, perhaps "control flow lifetimes" would be apt? |
This comment has been minimized.
This comment has been minimized.
I'd say open PRs or open issues on the nll-rfc repo. |
This comment has been minimized.
This comment has been minimized.
rfcbot
commented
Sep 18, 2017
|
|
rfcbot
added
final-comment-period
and removed
proposed-final-comment-period
labels
Sep 18, 2017
This comment has been minimized.
This comment has been minimized.
rfcbot
commented
Sep 28, 2017
|
The final comment period is now complete. |
aturon
referenced this pull request
Sep 29, 2017
Closed
Tracking issue for RFC #2094: non-lexical lifetimes #44928
This comment has been minimized.
This comment has been minimized.
|
|
aturon
merged commit aa80b68
into
rust-lang:master
Sep 29, 2017
This comment has been minimized.
This comment has been minimized.
|
This has been a long time coming! :D Congrats to Niko and the compiler team for all their hardwork on getting this right! |
This comment has been minimized.
This comment has been minimized.
00imvj00
commented
Oct 31, 2017
|
so, when it will be available in stable rust ? or nightly rust to play around with? @nikomatsakis |
This comment has been minimized.
This comment has been minimized.
golddranks
commented
Oct 31, 2017
|
@0freeman00 The approved RFCs have always a link to a tracking issue that has information about the implementation. Check this out: rust-lang/rust#44928 |
This comment has been minimized.
This comment has been minimized.
pravic
commented
Nov 9, 2017
|
The feature name is not filled which broke the RFC book a bit: https://rust-lang.github.io/rfcs/2094-nll.html (note |
nikomatsakis commentedAug 2, 2017
•
edited by mbrubeck
Extend Rust's borrow system to support non-lexical lifetimes -- these are lifetimes that are based on the control-flow graph, rather than lexical scopes. The RFC describes in detail how to infer these new, more flexible regions, and also describes how to adjust our error messages. The RFC also describes a few other extensions to the borrow checker, the total effect of which is to eliminate many common cases where small, function-local requires would be required to pass the borrow check. (The appendix describes some of the remaining borrow-checker limitations that are not addressed by this RFC.)
Due to its size and complexity, this RFC is being run through an experimental process. The text of the RFC itself is not in this file -- rather, it can be found at the non-lexical lifetimes repository. Prior to merging, the final version of the text will be added to this PR directly; until then, hosting the RFC at the repository allows for us to track (using open issues or pending PRs) important points of conversation and so forth. Feel free to leave ordinary comments on the PR, or to open issues -- important points will be elevated to issues for further discussion.
The ideas in this RFC have been implemented in prototype form. This prototype includes a simplified control-flow graph that allows one to create the various kinds of region constraints that can arise and implements the region inference algorithm which then solves those constraints.
Rendered.