Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dropped variables still included in generator type #57478

Open
Nemo157 opened this issue Jan 9, 2019 · 11 comments
Open

Dropped variables still included in generator type #57478

Nemo157 opened this issue Jan 9, 2019 · 11 comments

Comments

@Nemo157
Copy link
Contributor

@Nemo157 Nemo157 commented Jan 9, 2019

struct Foo;
impl !Send for Foo {}

let _: impl Send = || {
    let guard = Foo;
    drop(guard);
    yield;
};

(full playground) fails with

error[E0277]: `Foo` cannot be sent between threads safely
  --> src/main.rs:14:12
   |
14 |     let _: impl Send = || {
   |            ^^^^^^^^^ `Foo` cannot be sent between threads safely
   |
   = help: within `[generator@src/main.rs:14:24: 18:6 {Foo, ()}]`, the trait `std::marker::Send` is not implemented for `Foo`
   = note: required because it appears within the type `{Foo, ()}`
   = note: required because it appears within the type `[generator@src/main.rs:14:24: 18:6 {Foo, ()}]`

The guard should be dead and deallocated before the yield point so shouldn't appear in the generator type and affect the Sendness. Wrapping the guard in a new scope before the yield avoids this (included in the playground). First noticed in relation to async functions on u.rl.o.

@Aaron1011

This comment has been minimized.

Copy link
Contributor

@Aaron1011 Aaron1011 commented Jan 23, 2019

I'd like to work on this.

@Aaron1011

This comment has been minimized.

Copy link
Contributor

@Aaron1011 Aaron1011 commented Jan 23, 2019

This is going to be tricky.

The error is occurring due to the computed generator witness type including Foo. This is done during initial type checking, before any MIR has been generated.

Making this work would require type resolution to depend on the results of NLL. Specifically, the computed generator witness type would have to depend on which locals are computed to be live during mir-borrowck.

@Zoxc @nikomatsakis: Thoughts?

@Zoxc

This comment has been minimized.

Copy link
Contributor

@Zoxc Zoxc commented Jan 26, 2019

My plan for this is just to generate MIR for just the generator during type checking and then do the analysis on MIR. Currently that isn't very feasible given the current compiler structure.

Aaron1011 added a commit to Aaron1011/rust that referenced this issue Jun 7, 2019
Compound operators (e.g. 'a += b') have two different possible
evaluation orders. When the left-hand side is a primitive type, the
expression is evaluated right-to-left. However, when the left-hand side
is a non-primitive type, the expression is evaluated left-to-right.

This causes problems when we try to determine if a type is live across a
yield point. Since we need to perform this computation before typecheck
has run, we can't simply check the types of the operands.

This commit calculates the most 'pessimistic' scenario - that is,
erring on the side of treating more types as live, rather than fewer.
This is perfectly safe - in fact, this initial liveness computation is
already overly conservative (e.g. issue rust-lang#57478). The important thing is
that we compute a superset of the types that are actually live across
yield points. When we generate MIR, we'll determine which types actually
need to stay live across a given yield point, and which ones cam
actually be dropped.

Concretely, we force the computed HIR traversal index for
right-hand-side yield expression to be equal to the maximum index for
the left-hand side. This covers both possible execution orders:

* If the expression is evalauted right-to-left, our 'pessismitic' index
is unecessary, but safe. We visit the expressions in an
ExprKind::AssignOp from right to left, so it actually would have been
safe to do nothing. However, while increasing the index of a yield point
might cause the compiler to reject code that could actually compile, it
will never cause incorrect code to be accepted.
* If the expression is evaluated left-to-right, our 'pessimistic' index
correctly ensures that types in the left-hand-side are seen as occuring
before the yield - which is exactly what we want
@tmandry

This comment has been minimized.

Copy link
Contributor

@tmandry tmandry commented Jun 12, 2019

cc @rust-lang/lang, are we 100% sure we want to support this? The implication is there is going to be a "sharp edge" here when drops for certain locals move around relative to yield points (or vice versa).

It's also possible to work around it by turning your scope into a function call (possibly a closure that is immediately called).

@Nemo157

This comment has been minimized.

Copy link
Contributor Author

@Nemo157 Nemo157 commented Jun 12, 2019

You don't need a function call, just adding an actual scope around the variable that's dropped between yields is enough (this is included in the playground):

let _: impl Send = || {
    {
        let guard = Foo;
        drop(guard);
    }
    yield;
};

It makes sense to me why this is as it is and the solution could be just improved diagnostics telling users to add these extra scopes, it just seems like an unnecessary pain point for async functions.

Aaron1011 added a commit to Aaron1011/rust that referenced this issue Jun 22, 2019
Compound operators (e.g. 'a += b') have two different possible
evaluation orders. When the left-hand side is a primitive type, the
expression is evaluated right-to-left. However, when the left-hand side
is a non-primitive type, the expression is evaluated left-to-right.

This causes problems when we try to determine if a type is live across a
yield point. Since we need to perform this computation before typecheck
has run, we can't simply check the types of the operands.

This commit calculates the most 'pessimistic' scenario - that is,
erring on the side of treating more types as live, rather than fewer.
This is perfectly safe - in fact, this initial liveness computation is
already overly conservative (e.g. issue rust-lang#57478). The important thing is
that we compute a superset of the types that are actually live across
yield points. When we generate MIR, we'll determine which types actually
need to stay live across a given yield point, and which ones cam
actually be dropped.

Concretely, we force the computed HIR traversal index for
right-hand-side yield expression to be equal to the maximum index for
the left-hand side. This covers both possible execution orders:

* If the expression is evalauted right-to-left, our 'pessismitic' index
is unecessary, but safe. We visit the expressions in an
ExprKind::AssignOp from right to left, so it actually would have been
safe to do nothing. However, while increasing the index of a yield point
might cause the compiler to reject code that could actually compile, it
will never cause incorrect code to be accepted.
* If the expression is evaluated left-to-right, our 'pessimistic' index
correctly ensures that types in the left-hand-side are seen as occuring
before the yield - which is exactly what we want
@Nemo157

This comment has been minimized.

Copy link
Contributor Author

@Nemo157 Nemo157 commented Aug 21, 2019

After being reminded of this I realised this isn't really about "dropping" variables, it's about moving the variables out of the generator, any function that takes ownership should cause the variable to no longer be alive in the generator (this is the same thing since drop is just a trivial function to move the variable out, but being explicit about it might help others like me that didn't instantly make the connection):

struct Foo;
impl !Send for Foo {}

fn use_foo(_: Foo) {}

let _: impl Send = || {
    let guard = Foo;
    use_foo(guard);
    yield;
};
@oconnor663

This comment has been minimized.

Copy link
Contributor

@oconnor663 oconnor663 commented Aug 21, 2019

Another twist on the same problem might be using something like ManuallyDrop:

struct Foo;
impl !Send for Foo {}

let _: impl Send = || {
    let guard = ManuallyDrop::new(Foo);
    yield;
};
@tmandry

This comment has been minimized.

Copy link
Contributor

@tmandry tmandry commented Jan 21, 2020

Marking as AsyncAwait-OnDeck - this error can be confusing, and it may not be obvious how to work around it.

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

@nikomatsakis nikomatsakis commented Jan 24, 2020

What exactly is the bug here? To improve error report, or to do more precise drops? If the latter, that's a tricky problem indeed, but also duplicated by other issues.

@tmandry

This comment has been minimized.

Copy link
Contributor

@tmandry tmandry commented Feb 11, 2020

The way I read the issue, it is about tracking drops more precisely. (What issues duplicate this?)

We should open a separate issue to track improving the error message.

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

@nikomatsakis nikomatsakis commented Feb 24, 2020

@tmandry I'm not sure what issue is the "best duplicate" but I think we've been using #57017 as a kind of stand-in for "more precise generator captures". It'd probably be good to create a generalized tracking issue that dives into the different sorts of cases, since it doesn't look like we're likely to get a generalized fix in the near-ish term.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants
You can’t perform that action at this time.