Non-lexical borrow scopes and better treatment of nested method calls #811

Closed
pnkfelix opened this Issue Feb 5, 2015 · 41 comments

Comments

Projects
None yet
@pnkfelix
Member

pnkfelix commented Feb 5, 2015

In general, it can be useful to have support for borrows that aren't tied to lexical scopes.

Related links:

UPDATE: The relevant RFCs have been accepted. See rust-lang/rust#43234 to track implementation progress here.

@pnkfelix pnkfelix added the postponed label Feb 5, 2015

@nikomatsakis nikomatsakis changed the title from Single-entry / multiple-exit regions for borrows to Non-lexical borrow scopes and better treatment of nested method calls Apr 16, 2015

steveklabnik added a commit to steveklabnik/rust that referenced this issue Jun 12, 2015

Remove a bunch of ignored tests
Most of these are old, but some specific messages for specific tests:

* trait-contravariant-self.rs: failed due to a soundess hole:
  rust-lang@05e3248

* process-detatch: rust-lang@15966c3
  says "this test is being ignored until signals are implemented" That's
  not happening for a long time, and when it is, we'll write tests for
  it.

* deep-vector{,2}.rs: "too big for our poor macro infrastructure", and has
  been ignored over a year.

* borrowck-nested-calls.rs's FIXME #6268 was closed in favor of
  rust-lang/rfcs#811

* issue-15167.rs works properly now
* issue-9737.rs works properly now
* match-var-hygiene.rs works properly now

steveklabnik added a commit to steveklabnik/rust that referenced this issue Jun 12, 2015

ignore-test cleanup
Most of these are old, but some specific messages for specific tests:

* trait-contravariant-self.rs: failed due to a soundess hole:
  rust-lang@05e3248

* process-detatch: rust-lang@15966c3
  says "this test is being ignored until signals are implemented" That's
  not happening for a long time, and when it is, we'll write tests for
  it.

* deep-vector{,2}.rs: "too big for our poor macro infrastructure", and has
  been ignored over a year.

* borrowck-nested-calls.rs's FIXME #6268 was closed in favor of
  rust-lang/rfcs#811

* issue-15167.rs works properly now
* issue-9737.rs works properly now
* match-var-hygiene.rs works properly now

Addresses a chunk of #3965
@barosl

This comment has been minimized.

Show comment
Hide comment
@barosl

barosl Jun 24, 2015

Contributor

@mike-marcacci pointed out that the single-entry, multiple-exit regions issue also applies to the if let struct. While a code like

if let Some(val) = map.get_mut(&id) {
   /* ... */
} else {
   map.insert(id, new_val);
}

seems "innocent", it is actually just a syntax sugar for the match block, so the borrow lasts until the end of the else block. This particular case could be improved, or at least diagnostics could be, but I'm not sure fixing the specific point without introducing a general solution is a good idea.

Contributor

barosl commented Jun 24, 2015

@mike-marcacci pointed out that the single-entry, multiple-exit regions issue also applies to the if let struct. While a code like

if let Some(val) = map.get_mut(&id) {
   /* ... */
} else {
   map.insert(id, new_val);
}

seems "innocent", it is actually just a syntax sugar for the match block, so the borrow lasts until the end of the else block. This particular case could be improved, or at least diagnostics could be, but I'm not sure fixing the specific point without introducing a general solution is a good idea.

thepowersgang added a commit to thepowersgang/rust that referenced this issue Jul 25, 2015

ignore-test cleanup
Most of these are old, but some specific messages for specific tests:

* trait-contravariant-self.rs: failed due to a soundess hole:
  rust-lang@05e3248

* process-detatch: rust-lang@15966c3
  says "this test is being ignored until signals are implemented" That's
  not happening for a long time, and when it is, we'll write tests for
  it.

* deep-vector{,2}.rs: "too big for our poor macro infrastructure", and has
  been ignored over a year.

* borrowck-nested-calls.rs's FIXME #6268 was closed in favor of
  rust-lang/rfcs#811

* issue-15167.rs works properly now
* issue-9737.rs works properly now
* match-var-hygiene.rs works properly now

Addresses a chunk of #3965
@flavius

This comment has been minimized.

Show comment
Hide comment
@flavius

flavius Sep 13, 2015

What's the state of this feature? Any new plans and/or changes with this?

flavius commented Sep 13, 2015

What's the state of this feature? Any new plans and/or changes with this?

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Oct 1, 2015

Member

@flavius basically, the HIR/MIR work needs to be completed first.

Member

steveklabnik commented Oct 1, 2015

@flavius basically, the HIR/MIR work needs to be completed first.

@ebiggers

This comment has been minimized.

Show comment
Hide comment
@ebiggers

ebiggers Feb 21, 2016

I would like to see this implemented. I believe I ran into this with the following code which is descending a prefix tree and inserting new nodes:

struct TrieNode {
    bitmask: u32,
    terminal: bool,
    children: Vec<Box<TrieNode>>,
}

impl TrieNode {
    fn new() -> TrieNode {
        TrieNode {
            bitmask: 0,
            terminal: false,
            children: Vec::with_capacity(2),
        }
    }

    fn insert(&mut self, word: &Vec<u8>) {
        let mut node = self;
        for b in word {
            let bit = 1u32 << b;
            let index = (node.bitmask & (bit - 1)).count_ones() as usize;
            if (node.bitmask & bit) == 0 {
                node.children.insert(index, Box::new(TrieNode::new()));
                node.bitmask |= bit;
            }
            node = &mut node.children[index];
        }
        node.terminal = true;
    }
}

The problematic line is this one:

node = &mut node.children[index];

This causes the Rust compiler to spew a page of errors:

test.rs:22:17: 22:30 error: cannot borrow `node.children` as mutable more than once at a time [E0499]
test.rs:22                 node.children.insert(index, Box::new(TrieNode::new()));
               ^~~~~~~~~~~~~
test.rs:22:17: 22:30 help: run `rustc --explain E0499` to see a detailed explanation
test.rs:25:25: 25:38 note: previous borrow of `node.children` occurs here; the mutable borrow prevents subsequent moves, borrows, or modification of `node.children` until the borrow ends
test.rs:25             node = &mut node.children[index];
                   ^~~~~~~~~~~~~
test.rs:28:6: 28:6 note: previous borrow ends here
test.rs:16     fn insert(&mut self, word: &Vec<u8>) {
...
test.rs:28     }
           ^
test.rs:25:13: 25:45 error: cannot assign to `node` because it is borrowed [E0506]
test.rs:25             node = &mut node.children[index];
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test.rs:25:25: 25:38 note: borrow of `node` occurs here
test.rs:25             node = &mut node.children[index];
                   ^~~~~~~~~~~~~
test.rs:25:25: 25:38 error: cannot borrow `node.children` as mutable more than once at a time [E0499]
test.rs:25             node = &mut node.children[index];
                   ^~~~~~~~~~~~~
test.rs:25:25: 25:38 help: run `rustc --explain E0499` to see a detailed explanation
test.rs:25:25: 25:38 note: previous borrow of `node.children` occurs here; the mutable borrow prevents subsequent moves, borrows, or modification of `node.children` until the borrow ends
test.rs:25             node = &mut node.children[index];
                   ^~~~~~~~~~~~~
test.rs:28:6: 28:6 note: previous borrow ends here
test.rs:16     fn insert(&mut self, word: &Vec<u8>) {
...
test.rs:28     }

This can be "fixed" by replacing the line with this:

let tmp = node;
node = &mut tmp.children[index];

In my opinion this adds no value and simply serves as a means of appeasing the borrow checker. The program is correct either way; it seems to be purely a limitation of the borrow checker that it cannot determine this and the programmer has to explicitly add a temporary variable.

I understand there may be concern about making the borrow checker more complicated. However, I feel that usability takes precedence here. It's no secret that writing code the Rust compiler can prove is "safe" is one of the largest barriers to using Rust. Any change that makes the compiler smarter and able to correctly recognize more safe code as safe is, in my opinion, an improvement (at least, provided that it continues to follow well-defined rules).

I would like to see this implemented. I believe I ran into this with the following code which is descending a prefix tree and inserting new nodes:

struct TrieNode {
    bitmask: u32,
    terminal: bool,
    children: Vec<Box<TrieNode>>,
}

impl TrieNode {
    fn new() -> TrieNode {
        TrieNode {
            bitmask: 0,
            terminal: false,
            children: Vec::with_capacity(2),
        }
    }

    fn insert(&mut self, word: &Vec<u8>) {
        let mut node = self;
        for b in word {
            let bit = 1u32 << b;
            let index = (node.bitmask & (bit - 1)).count_ones() as usize;
            if (node.bitmask & bit) == 0 {
                node.children.insert(index, Box::new(TrieNode::new()));
                node.bitmask |= bit;
            }
            node = &mut node.children[index];
        }
        node.terminal = true;
    }
}

The problematic line is this one:

node = &mut node.children[index];

This causes the Rust compiler to spew a page of errors:

test.rs:22:17: 22:30 error: cannot borrow `node.children` as mutable more than once at a time [E0499]
test.rs:22                 node.children.insert(index, Box::new(TrieNode::new()));
               ^~~~~~~~~~~~~
test.rs:22:17: 22:30 help: run `rustc --explain E0499` to see a detailed explanation
test.rs:25:25: 25:38 note: previous borrow of `node.children` occurs here; the mutable borrow prevents subsequent moves, borrows, or modification of `node.children` until the borrow ends
test.rs:25             node = &mut node.children[index];
                   ^~~~~~~~~~~~~
test.rs:28:6: 28:6 note: previous borrow ends here
test.rs:16     fn insert(&mut self, word: &Vec<u8>) {
...
test.rs:28     }
           ^
test.rs:25:13: 25:45 error: cannot assign to `node` because it is borrowed [E0506]
test.rs:25             node = &mut node.children[index];
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test.rs:25:25: 25:38 note: borrow of `node` occurs here
test.rs:25             node = &mut node.children[index];
                   ^~~~~~~~~~~~~
test.rs:25:25: 25:38 error: cannot borrow `node.children` as mutable more than once at a time [E0499]
test.rs:25             node = &mut node.children[index];
                   ^~~~~~~~~~~~~
test.rs:25:25: 25:38 help: run `rustc --explain E0499` to see a detailed explanation
test.rs:25:25: 25:38 note: previous borrow of `node.children` occurs here; the mutable borrow prevents subsequent moves, borrows, or modification of `node.children` until the borrow ends
test.rs:25             node = &mut node.children[index];
                   ^~~~~~~~~~~~~
test.rs:28:6: 28:6 note: previous borrow ends here
test.rs:16     fn insert(&mut self, word: &Vec<u8>) {
...
test.rs:28     }

This can be "fixed" by replacing the line with this:

let tmp = node;
node = &mut tmp.children[index];

In my opinion this adds no value and simply serves as a means of appeasing the borrow checker. The program is correct either way; it seems to be purely a limitation of the borrow checker that it cannot determine this and the programmer has to explicitly add a temporary variable.

I understand there may be concern about making the borrow checker more complicated. However, I feel that usability takes precedence here. It's no secret that writing code the Rust compiler can prove is "safe" is one of the largest barriers to using Rust. Any change that makes the compiler smarter and able to correctly recognize more safe code as safe is, in my opinion, an improvement (at least, provided that it continues to follow well-defined rules).

@arielb1

This comment has been minimized.

Show comment
Hide comment
@arielb1

arielb1 Feb 21, 2016

Contributor

@ebiggers

Your issue here is reborrow vs. move, which is distinct from non-lexical borrows. You can also move node using &mut {node}.children[index] (see https://www.reddit.com/r/rust/comments/46rii1/when_to_borrow_move_or_copy/).

Contributor

arielb1 commented Feb 21, 2016

@ebiggers

Your issue here is reborrow vs. move, which is distinct from non-lexical borrows. You can also move node using &mut {node}.children[index] (see https://www.reddit.com/r/rust/comments/46rii1/when_to_borrow_move_or_copy/).

@ehiggs

This comment has been minimized.

Show comment
Hide comment
@ehiggs

ehiggs Mar 21, 2016

Is this related to places where matching an option won't compile using as_mut but can be fixed up by using Some(ref mut ...)?

e.g. code like this can fail because foo is still borrowed:

match foo.as_mut() {
    Some(x) => x.bar(), // fn bar(&mut self)
    None => foo = Some(Foo::new(7))
} 

It can be worked around like

match foo {
    Some(ref mut x) => x.bar(),
    None => foo = Some(Foo::new(7))
}

ehiggs commented Mar 21, 2016

Is this related to places where matching an option won't compile using as_mut but can be fixed up by using Some(ref mut ...)?

e.g. code like this can fail because foo is still borrowed:

match foo.as_mut() {
    Some(x) => x.bar(), // fn bar(&mut self)
    None => foo = Some(Foo::new(7))
} 

It can be worked around like

match foo {
    Some(ref mut x) => x.bar(),
    None => foo = Some(Foo::new(7))
}
@arielb1

This comment has been minimized.

Show comment
Hide comment
@arielb1

arielb1 Mar 22, 2016

Contributor

That's a different issue - non-lexical borrows.

Contributor

arielb1 commented Mar 22, 2016

That's a different issue - non-lexical borrows.

@ehiggs

This comment has been minimized.

Show comment
Hide comment
@ehiggs

ehiggs Mar 22, 2016

@arielb1, ok. Is there an issue for that? It seems like the two chunks of code I pasted should be semantically equivalent. @bluss on IRC pointed me to this one suggesting that it was similar to the issue I faced.

ehiggs commented Mar 22, 2016

@arielb1, ok. Is there an issue for that? It seems like the two chunks of code I pasted should be semantically equivalent. @bluss on IRC pointed me to this one suggesting that it was similar to the issue I faced.

@glaebhoerl

This comment has been minimized.

Show comment
Hide comment
@glaebhoerl

glaebhoerl Mar 22, 2016

Contributor

@arielb1 The title of this issue is "Non-lexical borrow scopes and"...

Contributor

glaebhoerl commented Mar 22, 2016

@arielb1 The title of this issue is "Non-lexical borrow scopes and"...

@bluss

This comment has been minimized.

Show comment
Hide comment
@bluss

bluss Mar 22, 2016

@ehiggs I wouldn't call that a work around, it's the idiomatic solution.

bluss commented Mar 22, 2016

@ehiggs I wouldn't call that a work around, it's the idiomatic solution.

@arielb1

This comment has been minimized.

Show comment
Hide comment
@arielb1

arielb1 Mar 22, 2016

Contributor

@glaebhoerl

Oops wrong RFC - I thought this was copy-vs-move.

Contributor

arielb1 commented Mar 22, 2016

@glaebhoerl

Oops wrong RFC - I thought this was copy-vs-move.

@aidanhs

This comment has been minimized.

Show comment
Hide comment
@aidanhs

aidanhs Jul 18, 2016

Member

Think this is related:

fn main() {
    let mut a = 1;
    let c;
    {
        let b = &mut a;
        c = &*b;
    }
    let d = &a;
}
cannot borrow `a` as immutable because it is also borrowed as mutable
Member

aidanhs commented Jul 18, 2016

Think this is related:

fn main() {
    let mut a = 1;
    let c;
    {
        let b = &mut a;
        c = &*b;
    }
    let d = &a;
}
cannot borrow `a` as immutable because it is also borrowed as mutable
@dbrgn

This comment has been minimized.

Show comment
Hide comment
@dbrgn

dbrgn Jul 19, 2016

@aidanhs no, I think that is correct. If you visualize the implicit scopes:

fn main() {
  let mut a = 1;
  { 
    let c;
    {
      let b = &mut a;
      {
        c = &*b;
      }
    }
    let d = &a;
  }
}

...then you can see that you're trying to borrow a while it's still borrowed mutably by b.

While borrowing a mutable reference to a value, that refrence is the only way to access that value at all.

dbrgn commented Jul 19, 2016

@aidanhs no, I think that is correct. If you visualize the implicit scopes:

fn main() {
  let mut a = 1;
  { 
    let c;
    {
      let b = &mut a;
      {
        c = &*b;
      }
    }
    let d = &a;
  }
}

...then you can see that you're trying to borrow a while it's still borrowed mutably by b.

While borrowing a mutable reference to a value, that refrence is the only way to access that value at all.

@droundy

This comment has been minimized.

Show comment
Hide comment
@droundy

droundy Jul 19, 2016

I thought the point of nonlexical borrows was precisely this kind of issue,
that it would be nice to have borrows last only as long as they are
actually used, not as long as the variable is in scope. In this case c is
never used after being assigned, so the borrow could be released.

On Tue, Jul 19, 2016, 12:14 AM Danilo Bargen notifications@github.com
wrote:

@aidanhs https://github.com/aidanhs no, I think that is correct. If you
visualize the implicit scopes:

fn main() {
let mut a = 1;
{
let c;
{
let b = &mut a;
{
c = &*b;
}
}
let d = &a;
}
}

...then you can see that you're trying to borrow a while it's still
borrowed mutably by b.

While borrowing a mutable reference to a value, that refrence is the only
way to access that value at all.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#811 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAIZKau_Sm7jskcN2AegvTS1I4QukMHOks5qXHk-gaJpZM4DceTV
.

droundy commented Jul 19, 2016

I thought the point of nonlexical borrows was precisely this kind of issue,
that it would be nice to have borrows last only as long as they are
actually used, not as long as the variable is in scope. In this case c is
never used after being assigned, so the borrow could be released.

On Tue, Jul 19, 2016, 12:14 AM Danilo Bargen notifications@github.com
wrote:

@aidanhs https://github.com/aidanhs no, I think that is correct. If you
visualize the implicit scopes:

fn main() {
let mut a = 1;
{
let c;
{
let b = &mut a;
{
c = &*b;
}
}
let d = &a;
}
}

...then you can see that you're trying to borrow a while it's still
borrowed mutably by b.

While borrowing a mutable reference to a value, that refrence is the only
way to access that value at all.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#811 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAIZKau_Sm7jskcN2AegvTS1I4QukMHOks5qXHk-gaJpZM4DceTV
.

@remram44

This comment has been minimized.

Show comment
Hide comment
@remram44

remram44 Jul 19, 2016

(@droundy) This is really an optimization issue and not a language one. You wrote the code and the compiler should very much check that it makes sense. You can't have an error spuriously appear when you change code somewhere down the line and this variable becomes actually used, that would be insane.

remram44 commented Jul 19, 2016

(@droundy) This is really an optimization issue and not a language one. You wrote the code and the compiler should very much check that it makes sense. You can't have an error spuriously appear when you change code somewhere down the line and this variable becomes actually used, that would be insane.

@eddyb

This comment has been minimized.

Show comment
Hide comment
@eddyb

eddyb Jul 19, 2016

Member

@remram44 Non-lexical borrows would still be deterministic, with no optimizations applied before the checks. The error wouldn't spurious, it would depend on a clearly visible use of the variable.

Member

eddyb commented Jul 19, 2016

@remram44 Non-lexical borrows would still be deterministic, with no optimizations applied before the checks. The error wouldn't spurious, it would depend on a clearly visible use of the variable.

@ehiggs

This comment has been minimized.

Show comment
Hide comment
@ehiggs

ehiggs Jul 19, 2016

Is it possible to have something like:

fn main() {
    let a = 1;
    let c;
    {
        let b = &mut a;
        c = SomethingThatHasSideEffectsOnDrop::new(&*b);
    }
    side_effect_function("called before c is dropped?");
    let d = &a;
}

Would c be dropped after last use, or just before the data it holds must be relinquished?

ehiggs commented Jul 19, 2016

Is it possible to have something like:

fn main() {
    let a = 1;
    let c;
    {
        let b = &mut a;
        c = SomethingThatHasSideEffectsOnDrop::new(&*b);
    }
    side_effect_function("called before c is dropped?");
    let d = &a;
}

Would c be dropped after last use, or just before the data it holds must be relinquished?

@e-oz

This comment has been minimized.

Show comment
Hide comment
@e-oz

e-oz Jul 19, 2016

@ehiggs c is declared outside of lexical scope so it should be released only after main function scope.

e-oz commented Jul 19, 2016

@ehiggs c is declared outside of lexical scope so it should be released only after main function scope.

@droundy

This comment has been minimized.

Show comment
Hide comment
@droundy

droundy Jul 19, 2016

@ehiggs I think this issue is only for non-lexical borrows, not for non-lexical lifetimes, which would effect the time of dropping. I doubt that non-lexical lifetimes will ever seriously be considered, because it would affect the semantics of tons of existing code.

droundy commented Jul 19, 2016

@ehiggs I think this issue is only for non-lexical borrows, not for non-lexical lifetimes, which would effect the time of dropping. I doubt that non-lexical lifetimes will ever seriously be considered, because it would affect the semantics of tons of existing code.

@eddyb

This comment has been minimized.

Show comment
Hide comment
@eddyb

eddyb Jul 19, 2016

Member

@ehiggs The drops would remain lexical and thus your example wouldn't compile (i.e. the drop of c at the end of main counts as an use of the borrow on a through b).

Member

eddyb commented Jul 19, 2016

@ehiggs The drops would remain lexical and thus your example wouldn't compile (i.e. the drop of c at the end of main counts as an use of the borrow on a through b).

@eddyb

This comment has been minimized.

Show comment
Hide comment
@eddyb

eddyb Jul 19, 2016

Member

@droundy Non-lexical lifetimes will exist (since borrows are based on lifetimes), what won't is non-lexical (drop) scopes. Variables that have a destructor will result in lexical lifetimes still, even with NLLs.

Member

eddyb commented Jul 19, 2016

@droundy Non-lexical lifetimes will exist (since borrows are based on lifetimes), what won't is non-lexical (drop) scopes. Variables that have a destructor will result in lexical lifetimes still, even with NLLs.

@e-oz

This comment has been minimized.

Show comment
Hide comment
@e-oz

e-oz Jul 19, 2016

after writing example I realized it would be total mess, worse than chaotic goto.
Maybe we really shouldn't change rules of borrow-checker...

e-oz commented Jul 19, 2016

after writing example I realized it would be total mess, worse than chaotic goto.
Maybe we really shouldn't change rules of borrow-checker...

@eddyb

This comment has been minimized.

Show comment
Hide comment
@eddyb

eddyb Jul 19, 2016

Member

@e-oz What do you mean? The rules are intended to be relatively intuitive ("I use this here and here and after that I'm done with it, cancel all borrows it held").

Member

eddyb commented Jul 19, 2016

@e-oz What do you mean? The rules are intended to be relatively intuitive ("I use this here and here and after that I'm done with it, cancel all borrows it held").

@e-oz

This comment has been minimized.

Show comment
Hide comment
@e-oz

e-oz Jul 19, 2016

@eddyb I think they are currently intuitive and manual declaration where to cancel borrows (aka "non-scope") can make them not only counter-intuitive but also dangerous and difficult to read and extend code.

e-oz commented Jul 19, 2016

@eddyb I think they are currently intuitive and manual declaration where to cancel borrows (aka "non-scope") can make them not only counter-intuitive but also dangerous and difficult to read and extend code.

@evestera

This comment has been minimized.

Show comment
Hide comment
@evestera

evestera Jul 19, 2016

If you care about this kind of thing and want to know how it may work I highly recommend @nikomatsakis series of blog posts on non-lexical lifetimes.

I think that the discussed changes really will make things more intuitive and very much worth it for the general increase in expressiveness as well. But at the same time it may seem more complex for the times when you really want to understand what is happening. Especially considering details like structs with destructors/drop effects working differently (i.e. like they do now).

In part I think this added complexity can be alleviated by good tooling support though (especially in the form of visualization, and I have some hopes and plans for that myself), but I think that is another discussion.

If you care about this kind of thing and want to know how it may work I highly recommend @nikomatsakis series of blog posts on non-lexical lifetimes.

I think that the discussed changes really will make things more intuitive and very much worth it for the general increase in expressiveness as well. But at the same time it may seem more complex for the times when you really want to understand what is happening. Especially considering details like structs with destructors/drop effects working differently (i.e. like they do now).

In part I think this added complexity can be alleviated by good tooling support though (especially in the form of visualization, and I have some hopes and plans for that myself), but I think that is another discussion.

@aidanhs

This comment has been minimized.

Show comment
Hide comment
@aidanhs

aidanhs Jul 20, 2016

Member

@dbrgn visualising it with scopes is just showing lexical borrows at work?

To clarify my example, I wanted to downgrade my &mut to a & for the remainder of the borrow. While borrow scopes are lexical this isn't possible because there's no way to say "stop the &mut borrow and replace with a &, permitting new & borrows from the original variable" (though there are workarounds).

Non-lexical borrows may not solve my problem, but they are a prerequisite for any possible fix I see, so I wanted to add it as an example to be considered (and rejected, if it's not useful enough).

(a follow-up discussion is the conditions under which things are allowed to be downgraded (e.g. struct fields downgrade allowing struct downgrade, ability to express relationship between arg types and return type downgrades in a function signature (related discussion)), but possibly something for another time)

Member

aidanhs commented Jul 20, 2016

@dbrgn visualising it with scopes is just showing lexical borrows at work?

To clarify my example, I wanted to downgrade my &mut to a & for the remainder of the borrow. While borrow scopes are lexical this isn't possible because there's no way to say "stop the &mut borrow and replace with a &, permitting new & borrows from the original variable" (though there are workarounds).

Non-lexical borrows may not solve my problem, but they are a prerequisite for any possible fix I see, so I wanted to add it as an example to be considered (and rejected, if it's not useful enough).

(a follow-up discussion is the conditions under which things are allowed to be downgraded (e.g. struct fields downgrade allowing struct downgrade, ability to express relationship between arg types and return type downgrades in a function signature (related discussion)), but possibly something for another time)

@ehiggs

This comment has been minimized.

Show comment
Hide comment
@ehiggs

ehiggs Jul 20, 2016

@evestera Thanks, that's a great blog series by @nikomatsakis.

@eddyb Thanks for explaining that this doesn't pertain to structs with drop defined. nikomatsakis' posts discuss how dropck is a different system from the borrowchecker so now it all makes sense.

ehiggs commented Jul 20, 2016

@evestera Thanks, that's a great blog series by @nikomatsakis.

@eddyb Thanks for explaining that this doesn't pertain to structs with drop defined. nikomatsakis' posts discuss how dropck is a different system from the borrowchecker so now it all makes sense.

@bluss bluss referenced this issue in rust-lang/rust Sep 12, 2016

Closed

Worse than lexical borrowing #36403

@ebkalderon

This comment has been minimized.

Show comment
Hide comment
@ebkalderon

ebkalderon Oct 17, 2016

Just curious, what is the current status of this issue? Will work on non-lexical borrows resume now that MIR has landed on Nightly?

Just curious, what is the current status of this issue? Will work on non-lexical borrows resume now that MIR has landed on Nightly?

@burdges burdges referenced this issue in droundy/arrayref Oct 26, 2016

Closed

Returning split up arrays by value #5

@oli-obk

This comment has been minimized.

Show comment
Hide comment
@oli-obk

oli-obk Nov 8, 2016

Contributor

Not sure if that was intentional or not, but beta and nightly allow the following piece of code to work (not panic at runtime):

use std::cell::RefCell;

fn main() {
    let mut data = RefCell::new(None);
    let inner = if data.borrow().is_none() {
        let d = 1337;
        *data.borrow_mut() = Some(d);
        d
    } else {
        data.borrow().unwrap()
    };
}

(found here: http://stackoverflow.com/questions/40482981/if-condition-remains-borrowed-in-body )

Contributor

oli-obk commented Nov 8, 2016

Not sure if that was intentional or not, but beta and nightly allow the following piece of code to work (not panic at runtime):

use std::cell::RefCell;

fn main() {
    let mut data = RefCell::new(None);
    let inner = if data.borrow().is_none() {
        let d = 1337;
        *data.borrow_mut() = Some(d);
        d
    } else {
        data.borrow().unwrap()
    };
}

(found here: http://stackoverflow.com/questions/40482981/if-condition-remains-borrowed-in-body )

@eddyb

This comment has been minimized.

Show comment
Hide comment
@eddyb

eddyb Nov 8, 2016

Member

It was intentional, the previous behavior was a long-standing bug that was only partially fixed before.

Member

eddyb commented Nov 8, 2016

It was intentional, the previous behavior was a long-standing bug that was only partially fixed before.

@radix

This comment has been minimized.

Show comment
Hide comment
@radix

radix Dec 2, 2016

Can this ticket be closed? I see that all three of the linked issues in the description are closed. If it still needs to be open, is it also still appropriate to have the "postponed" label?

radix commented Dec 2, 2016

Can this ticket be closed? I see that all three of the linked issues in the description are closed. If it still needs to be open, is it also still appropriate to have the "postponed" label?

@istankovic

This comment has been minimized.

Show comment
Hide comment
@istankovic

istankovic Dec 2, 2016

@radix The issue #29775 was linked to this one and closed at some point, however, I've just checked and it is still relevant. So if this one is to be closed, I suggest to reopen #29775 since it is stil relevant and represents a very basic use case.

@radix The issue #29775 was linked to this one and closed at some point, however, I've just checked and it is still relevant. So if this one is to be closed, I suggest to reopen #29775 since it is stil relevant and represents a very basic use case.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Dec 2, 2016

Member

Can this ticket be closed?

Why would we close it?

I see that all three of the linked issues in the description are closed.

That's because they're duplicates of this one.

If it still needs to be open, is it also still appropriate to have the "postponed" label?

Yes, as generally this means "we considered it but decided that it's not the right time yet." This is still true, as porting the borrowck to MIR is still a blocker.

Member

steveklabnik commented Dec 2, 2016

Can this ticket be closed?

Why would we close it?

I see that all three of the linked issues in the description are closed.

That's because they're duplicates of this one.

If it still needs to be open, is it also still appropriate to have the "postponed" label?

Yes, as generally this means "we considered it but decided that it's not the right time yet." This is still true, as porting the borrowck to MIR is still a blocker.

@radix

This comment has been minimized.

Show comment
Hide comment
@radix

radix Dec 2, 2016

@steveklabnik I apologize. I was just unclear on the status since recent comments seemed to indicate it, or some parts of it, had been fixed. I also didn't realize that the links in the description were duplicates, I assumed they were dependents. My error. I should have worded my message more carefully. I didn't mean to say that I think it should be closed, just that I was curious about its status.

Thank you for the information.

@istankovic I'm not sure which #29775 you are referring to, as I can't find a reference to that number on this page.

radix commented Dec 2, 2016

@steveklabnik I apologize. I was just unclear on the status since recent comments seemed to indicate it, or some parts of it, had been fixed. I also didn't realize that the links in the description were duplicates, I assumed they were dependents. My error. I should have worded my message more carefully. I didn't mean to say that I think it should be closed, just that I was curious about its status.

Thank you for the information.

@istankovic I'm not sure which #29775 you are referring to, as I can't find a reference to that number on this page.

@istankovic

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Dec 2, 2016

Member

@radix it's all good! This is a very long-running and complicated issue, it's very easy to have done that. 😄

Member

steveklabnik commented Dec 2, 2016

@radix it's all good! This is a very long-running and complicated issue, it's very easy to have done that. 😄

@nikomatsakis

This comment has been minimized.

Show comment
Hide comment
@nikomatsakis

nikomatsakis Jan 10, 2017

Contributor

A rather long write-up about nested method calls specifically:

https://internals.rust-lang.org/t/accepting-nested-method-calls-with-an-mut-self-receiver/4588

Contributor

nikomatsakis commented Jan 10, 2017

A rather long write-up about nested method calls specifically:

https://internals.rust-lang.org/t/accepting-nested-method-calls-with-an-mut-self-receiver/4588

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Jun 27, 2017

Contributor

porting the borrowck to MIR is still a blocker.

What’s the status on this?


As far as I can tell, borrows being lexical means not only that some programs need to be more verbose, but also that some programs are impossible (at least with the desired API/characteristics). For example, I just tried to write something like this (reduced here):

pub enum Token {
    Whitespace,
    Ident(String),
    Number(i32),
}

pub struct Parser {
    /* ... */
}

impl Parser {
    pub fn advance_without_skipping_whitespace(&mut self) -> &Token {
        unimplemented!()
    }

    pub fn advance(&mut self) -> &Token {
        loop {
            match self.advance_without_skipping_whitespace() {
                &Token::Whitespace => {}
                result => return result
            }
        }
    }
}

Playpen

… and got this error message:

error[E0499]: cannot borrow `*self` as mutable more than once at a time
  --> <anon>:18:19
   |
18 |             match self.advance_without_skipping_whitespace() {
   |                   ^^^^
   |                   |
   |                   second mutable borrow occurs here
   |                   first mutable borrow occurs here
...
23 |     }
   |     - first borrow ends here

error: aborting due to previous error

The problem is the "first" borrow lasting for the whole function rather than just one iteration of the loop. I think that’s because the borrowed value is (in some code paths) returned, and borrow are lexical? I can’t find a way to rewrite this that compiles in today’s Rust, even tail recursion has the same problem as a loop.

Contributor

SimonSapin commented Jun 27, 2017

porting the borrowck to MIR is still a blocker.

What’s the status on this?


As far as I can tell, borrows being lexical means not only that some programs need to be more verbose, but also that some programs are impossible (at least with the desired API/characteristics). For example, I just tried to write something like this (reduced here):

pub enum Token {
    Whitespace,
    Ident(String),
    Number(i32),
}

pub struct Parser {
    /* ... */
}

impl Parser {
    pub fn advance_without_skipping_whitespace(&mut self) -> &Token {
        unimplemented!()
    }

    pub fn advance(&mut self) -> &Token {
        loop {
            match self.advance_without_skipping_whitespace() {
                &Token::Whitespace => {}
                result => return result
            }
        }
    }
}

Playpen

… and got this error message:

error[E0499]: cannot borrow `*self` as mutable more than once at a time
  --> <anon>:18:19
   |
18 |             match self.advance_without_skipping_whitespace() {
   |                   ^^^^
   |                   |
   |                   second mutable borrow occurs here
   |                   first mutable borrow occurs here
...
23 |     }
   |     - first borrow ends here

error: aborting due to previous error

The problem is the "first" borrow lasting for the whole function rather than just one iteration of the loop. I think that’s because the borrowed value is (in some code paths) returned, and borrow are lexical? I can’t find a way to rewrite this that compiles in today’s Rust, even tail recursion has the same problem as a loop.

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Jun 27, 2017

Contributor

some programs are impossible

That may have been a bit over-dramatic, I found a work-around. (Not returning the result of advance_without_skipping_whitespace(), using break instead of return, and than returning self.current_token() outside the loop.)

But the same kind of issues happen all the time when porting existing parsing code to this new API. Enough that this approach is not viable, I may have to do somewhat-expensive cloning everywhere.

Contributor

SimonSapin commented Jun 27, 2017

some programs are impossible

That may have been a bit over-dramatic, I found a work-around. (Not returning the result of advance_without_skipping_whitespace(), using break instead of return, and than returning self.current_token() outside the loop.)

But the same kind of issues happen all the time when porting existing parsing code to this new API. Enough that this approach is not viable, I may have to do somewhat-expensive cloning everywhere.

@pnkfelix

This comment has been minimized.

Show comment
Hide comment
@pnkfelix

pnkfelix Jun 27, 2017

Member

porting the borrowck to MIR is still a blocker.

What’s the status on this?

It is in-progress. Some steps have recently landed, but its not 100% working yet.

Member

pnkfelix commented Jun 27, 2017

porting the borrowck to MIR is still a blocker.

What’s the status on this?

It is in-progress. Some steps have recently landed, but its not 100% working yet.

@nikomatsakis

This comment has been minimized.

Show comment
Hide comment
@nikomatsakis

nikomatsakis Jan 25, 2018

Contributor

I think we can basically close this in favor of rust-lang/rust#43234

Contributor

nikomatsakis commented Jan 25, 2018

I think we can basically close this in favor of rust-lang/rust#43234

@petrochenkov petrochenkov added T-lang and removed postponed labels Feb 24, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment