Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NLL] Bad higher ranked subtype error #57374

Open
matthewjasper opened this issue Jan 6, 2019 · 34 comments
Open

[NLL] Bad higher ranked subtype error #57374

matthewjasper opened this issue Jan 6, 2019 · 34 comments
Assignees
Labels
A-NLL Area: Non Lexical Lifetimes (NLL) NLL-diagnostics Working torwads the "diagnostic parity" goal P-medium Medium priority T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@matthewjasper
Copy link
Contributor

matthewjasper commented Jan 6, 2019

With the removal of the leak check the MIR type checker is now responsible for reporting higher-ranked lifetime errors in full NLL mode. The error messages are not currently very helpful, since they weren't user visible until now.

The following code (play):

#![feature(nll)]

fn main() {
    let x: fn(&'static ()) = |_| {};
    let y: for<'a> fn(&'a ()) = x;
}

gives the following output

error: higher-ranked subtype error
  --> <source>:19:12
   |
19 |     let x: fn(&'static ()) = |_| {};
   |            ^^^^^^^^^^^^^^^
error: aborting due to previous error
Compiler returned: 1

In migrate mode or with AST borrowck the error is much clearer:

error[E0308]: mismatched types
  --> <source>:20:33
   |
20 |     let y: for<'a> fn(&'a ()) = x;
   |                                 ^ one type is more general than the other
   |
   = note: expected type `for<'a> fn(&'a ())`
              found type `fn(&'static ())`
error: aborting due to previous error
For more information about this error, try `rustc --explain E0308`.
Compiler returned: 1

cc @rust-lang/wg-compiler-nll

@matthewjasper matthewjasper added A-NLL Area: Non Lexical Lifetimes (NLL) NLL-diagnostics Working torwads the "diagnostic parity" goal labels Jan 6, 2019
@nikomatsakis
Copy link
Contributor

Yeah, I intended to port the newer errors to NLL but didn't do it yet since they are hidden by migration mode. But clearly we need to do that before we can stabilize NLL further.

@nikomatsakis nikomatsakis added I-nominated T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jan 7, 2019
@nikomatsakis
Copy link
Contributor

Nominating for discussion in the NLL meeting -- this should probably be prioritized.

@nikomatsakis nikomatsakis added P-high High priority and removed I-nominated labels Jan 9, 2019
@nikomatsakis nikomatsakis self-assigned this Jan 9, 2019
@nikomatsakis
Copy link
Contributor

Discussed in the NLL meeting. Assigning to @lqd but also to me to leave some mentoring notes and/or sync with @lqd

@pnkfelix
Copy link
Member

somewhat related bug: #57362

@pnkfelix
Copy link
Member

N.B.: This bug continues to persist even after Issue #57362 was resolved by PR #57901

@lqd
Copy link
Member

lqd commented Feb 27, 2019

With the temporary return of the leak-check from #58592, this issue has expectedly "un-regressed" to diagnostics levels very similar to pre-Universes or AST borrowck:

error[E0308]: mismatched types
 --> src/main.rs:5:33
  |
5 |     let y: for<'a> fn(&'a ()) = x;
  |                                 ^ expected concrete lifetime, found bound lifetime parameter 'a
  |
  = note: expected type `for<'a> fn(&'a ())`
             found type `fn(&'static ())`

@pnkfelix
Copy link
Member

As discussed in the NLL meeting last night, one can use -Z no-leak-check to skip the leak check and thus return to the (regressed) behavior and thus attempt to fix it properly.

@lqd
Copy link
Member

lqd commented Mar 6, 2019

As mentioned on Zulip, I have made some progress here (and have rambling/meandering notes from the exploratory analysis), but will need some guidance to continue, so unassigning myself until then.

@pnkfelix pnkfelix unassigned lqd Mar 6, 2019
@pnkfelix
Copy link
Member

triage: downgrading to P-medium.

Blocks: #59490

@pnkfelix pnkfelix added P-medium Medium priority and removed P-high High priority labels Mar 28, 2019
Aaron1011 added a commit to Aaron1011/rust that referenced this issue Oct 2, 2019
As described in rust-lang#57374, NLL currently produces unhelpful higher-ranked
trait bound (HRTB) errors when '-Zno-leak-check' is enabled.

This PR tackles one half of this issue - making the error message point
at the proper span. The error message itself is still the very generic
"higher-ranked subtype error", but this can be improved in a follow-up
PR.

The root cause of the bad spans lies in how NLL attempts to compute the
'blamed' region, for which it will retrieve a span for.
Consider the following code, which (correctly) does not compile:

```rust
let my_val: u8 = 25;
let a: &u8 = &my_val;
let b = a;
let c = b;
let d: &'static u8 = c;
```

This will cause NLL to generate the following subtype constraints:

d :< c
c :< b
b <: a

Since normal Rust lifetimes are covariant, this results in the following
region constraints (I'm using 'd to denote the lifetime of 'd',
'c to denote the lifetime of 'c, etc.):

'c: 'd
'b: 'c
'a: 'b

From this, we can derive that 'a: 'd holds, which implies that 'a: 'static
must hold. However, this is not the case, since 'a refers to 'my_val',
which does not outlive the current function.

When NLL attempts to infer regions for this code, it will see that the
region 'a has grown 'too large' - it will be inferred to outlive
'static, despite the fact that is not declared as outliving 'static
We can find the region responsible, 'd, by starting at the *end* of
the 'constraint chain' we generated above. This works because for normal
(non-higher-ranked) lifetimes, we generally build up a 'chain' of
lifetime constraints *away* from the original variable/lifetime.
That is, our original lifetime 'a is required to outlive progressively
more regions. If it ends up living for too long, we can look at the
'end' of this chain to determine the 'most recent' usage that caused
the lifetime to grow too large.

However, this logic does not work correctly when higher-ranked trait
bounds (HRTBs) come into play. This is because HRTBs have
*contravariance* with respect to their bound regions. For example,
this code snippet compiles:

```rust
let a: for<'a> fn(&'a ()) = |_| {};
let b: fn(&'static ()) = a;
```

Here, we require that 'a' is a subtype of 'b'. Because of
contravariance, we end up with the region constraint 'static: 'a,
*not* 'a: 'static

This means that our 'constraint chains' grow in the opposite direction
of 'normal lifetime' constraint chains. As we introduce subtypes, our
lifetime ends up being outlived by other lifetimes, rather than
outliving other lifetimes. Therefore, starting at the end of the
'constraint chain' will cause us to 'blame' a lifetime close to the original
definition of a variable, instead of close to where the bad lifetime
constraint is introduced.

This PR improves how we select the region to blame for 'too large'
universal lifetimes, when bound lifetimes are involved. If the region
we're checking is a 'placeholder' region (e.g. the region 'a' in
for<'a>, or the implicit region in fn(&())), we start traversing the
constraint chain from the beginning, rather than the end.

There are two (maybe more) different ways we generate region constraints for NLL:
requirements generated from trait queries, and requirements generated
from MIR subtype constraints. While the former always use explicit
placeholder regions, the latter is more tricky. In order to implement
contravariance for HRTBs, TypeRelating replaces placeholder regions with
existential regions. This requires us to keep track of whether or not an
existential region was originally a placeholder region. When we look for
a region to blame, we check if our starting region is either a
placeholder region or is an existential region created from a
placeholder region. If so, we start iterating from the beginning of the
constraint chain, rather than the end.
Centril added a commit to Centril/rust that referenced this issue Oct 3, 2019
…akis

Improve HRTB error span when -Zno-leak-check is used

As described in rust-lang#57374, NLL currently produces unhelpful higher-ranked
trait bound (HRTB) errors when '-Zno-leak-check' is enabled.

This PR tackles one half of this issue - making the error message point
at the proper span. The error message itself is still the very generic
"higher-ranked subtype error", but this can be improved in a follow-up
PR.

The root cause of the bad spans lies in how NLL attempts to compute the
'blamed' region, for which it will retrieve a span for.
Consider the following code, which (correctly) does not compile:

```rust
let my_val: u8 = 25;
let a: &u8 = &my_val;
let b = a;
let c = b;
let d: &'static u8 = c;
```

This will cause NLL to generate the following subtype constraints:

d :< c
c :< b
b <: a

Since normal Rust lifetimes are covariant, this results in the following
region constraints (I'm using 'd to denote the lifetime of 'd',
'c to denote the lifetime of 'c, etc.):

'c: 'd
'b: 'c
'a: 'b

From this, we can derive that 'a: 'd holds, which implies that 'a: 'static
must hold. However, this is not the case, since 'a refers to 'my_val',
which does not outlive the current function.

When NLL attempts to infer regions for this code, it will see that the
region 'a has grown 'too large' - it will be inferred to outlive
'static, despite the fact that is not declared as outliving 'static
We can find the region responsible, 'd, by starting at the *end* of
the 'constraint chain' we generated above. This works because for normal
(non-higher-ranked) lifetimes, we generally build up a 'chain' of
lifetime constraints *away* from the original variable/lifetime.
That is, our original lifetime 'a is required to outlive progressively
more regions. If it ends up living for too long, we can look at the
'end' of this chain to determine the 'most recent' usage that caused
the lifetime to grow too large.

However, this logic does not work correctly when higher-ranked trait
bounds (HRTBs) come into play. This is because HRTBs have
*contravariance* with respect to their bound regions. For example,
this code snippet compiles:

```rust
let a: for<'a> fn(&'a ()) = |_| {};
let b: fn(&'static ()) = a;
```

Here, we require that 'a' is a subtype of 'b'. Because of
contravariance, we end up with the region constraint 'static: 'a,
*not* 'a: 'static

This means that our 'constraint chains' grow in the opposite direction
of 'normal lifetime' constraint chains. As we introduce subtypes, our
lifetime ends up being outlived by other lifetimes, rather than
outliving other lifetimes. Therefore, starting at the end of the
'constraint chain' will cause us to 'blame' a lifetime close to the original
definition of a variable, instead of close to where the bad lifetime
constraint is introduced.

This PR improves how we select the region to blame for 'too large'
universal lifetimes, when bound lifetimes are involved. If the region
we're checking is a 'placeholder' region (e.g. the region 'a' in
for<'a>, or the implicit region in fn(&())), we start traversing the
constraint chain from the beginning, rather than the end.

There are two (maybe more) different ways we generate region constraints for NLL:
requirements generated from trait queries, and requirements generated
from MIR subtype constraints. While the former always use explicit
placeholder regions, the latter is more tricky. In order to implement
contravariance for HRTBs, TypeRelating replaces placeholder regions with
existential regions. This requires us to keep track of whether or not an
existential region was originally a placeholder region. When we look for
a region to blame, we check if our starting region is either a
placeholder region or is an existential region created from a
placeholder region. If so, we start iterating from the beginning of the
constraint chain, rather than the end.
@nnethercote
Copy link
Contributor

I hit this today working on the compiler, calling a function with this signature:

pub fn link_name<F>(check_name: F, attrs: &[ast::Attribute]) -> Option<Symbol>
where
    F: Fn(&ast::Attribute, Symbol) -> bool

I had no idea what "Bad higher ranked subtype error" meant, so I googled and found this issue, and concluded it was somehow related to lifetimes. After some experimentation I got this signature to work:

pub fn link_name<'a, F>(check_name: F, attrs: &'a [ast::Attribute]) -> Option<Symbol>
where
    F: Fn(&'a ast::Attribute, Symbol) -> bool

So it was a happy ending, but there was some wailing and gnashing of teeth along the way.

@Aaron1011
Copy link
Member

With #88270 merged, the only test cases that still cause a higher ranked subtype error all use unstable features / Polonious. Are there any remaining cases where we get that message with stable code?

@lqd
Copy link
Member

lqd commented Aug 28, 2021

None that I'm personally aware of.

Note that the TAIT test triggering the higher-ranked subtype error is really triggering it, whereas the polonius ones were likely fixed by the 2 previous PRs: these blessed expectations haven't been updated yet (and CI doesn't validate them as of yet).

This issue is probably the closest/only issue tracking this specific diagnostic, so we could still use it to track the rest of the work that they need, or close it and open new ones.

From #86700 you can see that there are at least a few remaining tasks:

  • matching non-NLL output in these error messages, as requested by @estebank here. This probably will require passing these obligations and causes around. Otherwise we won't be able to be as precise as the current output and it would regress the diagnostics too much. There are a number of these cases Esteban pointed out during review.
  • the parity to non-NLL is also needed in some of the names currently used in the HRTB errors, for example this uses synthetic names for the regions, whereas non-NLL output uses the names from user code. It's likely that fixing the point above and having the correct obligations would also fix this though.
  • (some clean-up of the implementation is also required, but that's more of a task for enabling full-NLLs rather than this issue's errors)

@Aaron1011
Copy link
Member

matching non-NLL output in these error messages, as requested by @estebank here. This probably will require passing these obligations and causes around. Otherwise we won't be able to be as precise as the current output and it would regress the diagnostics too much. There are a number of these cases Esteban pointed out during review.

I'll start working on that

Aaron1011 added a commit to Aaron1011/rust that referenced this issue Sep 9, 2021
In some cases, we emit borrowcheck diagnostics pointing
at a particular field expression in a struct expression
(e.g. `MyStruct { field: my_expr }`). However, this
behavior currently relies on us choosing the
`ConstraintCategory::Boring` with the 'correct' span.
When adding additional variants to `ConstraintCategory`,
(or changing existing usages away from `ConstraintCategory::Boring`),
the current behavior can easily get broken, since a non-boring
constraint will get chosen over a boring one.

To make the diagnostic output less fragile, this commit
adds a `ConstraintCategory::Usage` variant. We use this variant
for the temporary assignments created for each field of
an aggregate we are constructing.

Using this new variant, we can emit a message mentioning
"this usage", emphasizing the fact that the error message
is related to the specific use site (in the struct expression).

This is preparation for additional work on improving NLL error messages
(see rust-lang#57374)
Manishearth added a commit to Manishearth/rust that referenced this issue Sep 16, 2021
Add `ConstraintCategory::Usage` for handling aggregate construction

In some cases, we emit borrowcheck diagnostics pointing
at a particular field expression in a struct expression
(e.g. `MyStruct { field: my_expr }`). However, this
behavior currently relies on us choosing the
`ConstraintCategory::Boring` with the 'correct' span.
When adding additional variants to `ConstraintCategory`,
(or changing existing usages away from `ConstraintCategory::Boring`),
the current behavior can easily get broken, since a non-boring
constraint will get chosen over a boring one.

To make the diagnostic output less fragile, this commit
adds a `ConstraintCategory::Usage` variant. We use this variant
for the temporary assignments created for each field of
an aggregate we are constructing.

Using this new variant, we can emit a message mentioning
"this usage", emphasizing the fact that the error message
is related to the specific use site (in the struct expression).

This is preparation for additional work on improving NLL error messages
(see rust-lang#57374)
GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this issue Sep 16, 2021
Add `ConstraintCategory::Usage` for handling aggregate construction

In some cases, we emit borrowcheck diagnostics pointing
at a particular field expression in a struct expression
(e.g. `MyStruct { field: my_expr }`). However, this
behavior currently relies on us choosing the
`ConstraintCategory::Boring` with the 'correct' span.
When adding additional variants to `ConstraintCategory`,
(or changing existing usages away from `ConstraintCategory::Boring`),
the current behavior can easily get broken, since a non-boring
constraint will get chosen over a boring one.

To make the diagnostic output less fragile, this commit
adds a `ConstraintCategory::Usage` variant. We use this variant
for the temporary assignments created for each field of
an aggregate we are constructing.

Using this new variant, we can emit a message mentioning
"this usage", emphasizing the fact that the error message
is related to the specific use site (in the struct expression).

This is preparation for additional work on improving NLL error messages
(see rust-lang#57374)
Aaron1011 added a commit to Aaron1011/rust that referenced this issue Sep 16, 2021
In some cases, we emit borrowcheck diagnostics pointing
at a particular field expression in a struct expression
(e.g. `MyStruct { field: my_expr }`). However, this
behavior currently relies on us choosing the
`ConstraintCategory::Boring` with the 'correct' span.
When adding additional variants to `ConstraintCategory`,
(or changing existing usages away from `ConstraintCategory::Boring`),
the current behavior can easily get broken, since a non-boring
constraint will get chosen over a boring one.

To make the diagnostic output less fragile, this commit
adds a `ConstraintCategory::Usage` variant. We use this variant
for the temporary assignments created for each field of
an aggregate we are constructing.

Using this new variant, we can emit a message mentioning
"this usage", emphasizing the fact that the error message
is related to the specific use site (in the struct expression).

This is preparation for additional work on improving NLL error messages
(see rust-lang#57374)
bors added a commit to rust-lang-ci/rust that referenced this issue Sep 20, 2021
Add `ConstraintCategory::Usage` for handling aggregate construction

In some cases, we emit borrowcheck diagnostics pointing
at a particular field expression in a struct expression
(e.g. `MyStruct { field: my_expr }`). However, this
behavior currently relies on us choosing the
`ConstraintCategory::Boring` with the 'correct' span.
When adding additional variants to `ConstraintCategory`,
(or changing existing usages away from `ConstraintCategory::Boring`),
the current behavior can easily get broken, since a non-boring
constraint will get chosen over a boring one.

To make the diagnostic output less fragile, this commit
adds a `ConstraintCategory::Usage` variant. We use this variant
for the temporary assignments created for each field of
an aggregate we are constructing.

Using this new variant, we can emit a message mentioning
"this usage", emphasizing the fact that the error message
is related to the specific use site (in the struct expression).

This is preparation for additional work on improving NLL error messages
(see rust-lang#57374)
@Aaron1011
Copy link
Member

I've opened #89249 to improve the cause information for some NLL error messages.

@Aaron1011
Copy link
Member

the parity to non-NLL is also needed in some of the names currently used in the HRTB errors, for example this uses synthetic names for the regions, whereas non-NLL output uses the names from user code. It's likely that fixing the point above and having the correct obligations would also fix this though.

I've opened #89250 to address this. It turned out to be caused by bound region anonimization during typeck.

@Aaron1011
Copy link
Member

Progress update:

From #86700 you can see that there are at least a few remaining tasks:

matching non-NLL output in these error messages, as requested by @estebank here. This probably will require passing these obligations and causes around. Otherwise we won't be able to be as precise as the current output and it would regress the diagnostics too much. There are a number of these cases Esteban pointed out during review.

PR #89249 has been merged, which makes NLL mode display the note that we display in non-NLL mode. I left comments on @estebank 's other requests in that PR - both of them are issues in non-NLL mode as well, so they're not directly related to this issue.

the parity to non-NLL is also needed in some of the names currently used in the HRTB errors, for example this uses synthetic names for the regions, whereas non-NLL output uses the names from user code. It's likely that fixing the point above and having the correct obligations would also fix this though.

This was addressed in PR #89250

(some clean-up of the implementation is also required, but that's more of a task for enabling full-NLLs rather than this issue's errors)

This still needs to be done, but it's not currently affecting the user-visible output

@Aaron1011
Copy link
Member

Are there any other user-visible diagnostic issues that need addressing?

@lqd
Copy link
Member

lqd commented Oct 1, 2021

On stable: I don't think so. It's also unlikely there are other open issues about them.

On nightly: like last time, some TAIT uses trigger the bad higher ranked subtype errors. Since Oli is currently revamping that, and it's unstable, there's no urgency to handle these IMO.

@Aaron1011
Copy link
Member

Issue #89620 hits a higher-ranked subtype error with #![feature(nll)]:

#![feature(nll)]

fn bar<Input>() -> impl Fn(Input) {
    |i| todo!()
}
    
fn foo() -> impl Fn(&str) {
    bar()
}

produces:

error: higher-ranked subtype error
 --> src/lib.rs:8:5
  |
8 |     bar()
  |     ^^^^^

error: could not compile `playground` due to previous error

@Aaron1011
Copy link
Member

Aaron1011 commented Dec 27, 2021

I've opened #92306 to address the issue with stable uses of opaque types

@marmeladema
Copy link
Contributor

marmeladema commented Mar 31, 2022

The error message is now identical between stable and nightly with nll enabled:

error[E0308]: mismatched types
 --> src/main.rs:5:33
  |
5 |     let y: for<'a> fn(&'a ()) = x;
  |                                 ^ one type is more general than the other
  |
  = note: expected fn pointer `for<'a> fn(&'a ())`
             found fn pointer `fn(&())`

For more information about this error, try `rustc --explain E0308`.

Maybe this issue can now be closed?

@estebank
Copy link
Contributor

The current output for aaron's case above

With nll

error: implementation of `Fn` is not general enough
 --> src/lib.rs:8:5
  |
8 |     bar()
  |     ^^^^^ implementation of `Fn` is not general enough
  |
  = note: `impl Fn(&'2 str)` must implement `Fn<(&'1 str,)>`, for any lifetime `'1`...
  = note: ...but it actually implements `Fn<(&'2 str,)>`, for some specific lifetime `'2`

error[E0308]: mismatched types
 --> src/lib.rs:8:5
  |
3 | fn bar<Input>() -> impl Fn(Input) {
  |                    --------------
  |                    |
  |                    the expected opaque type
  |                    the found opaque type
...
8 |     bar()
  |     ^^^^^ one type is more general than the other
  |
  = note: expected associated type `<impl Fn(&str) as FnOnce<(&str,)>>::Output`
             found associated type `<impl Fn(&str) as FnOnce<(&str,)>>::Output`

and without:

error: implementation of `Fn` is not general enough
 --> src/lib.rs:7:27
  |
7 |   fn foo() -> impl Fn(&str) {
  |  ___________________________^
8 | |     bar()
9 | | }
  | |_^ implementation of `Fn` is not general enough
  |
  = note: `impl Fn(&'2 str)` must implement `Fn<(&'1 str,)>`, for any lifetime `'1`...
  = note: ...but it actually implements `Fn<(&'2 str,)>`, for some specific lifetime `'2`

@marmeladema
Copy link
Contributor

@estebank I am sorry, but it's not obvious to me which one is better?

@Aaron1011
Copy link
Member

Aaron1011 commented Apr 20, 2022

I think the 'error: implementation of Fn is not general enough' message is better with NLL, since we point to the location responsible for producing the hidden type, instead of pointing at the entire function. However, the 'mismatched types' error is confusing and redundant.

@jackh726
Copy link
Member

@estebank
Copy link
Contributor

estebank commented May 24, 2022

I think that

  = note: `impl Fn(&'2 str)` must implement `Fn<(&'1 str,)>`, for any lifetime `'1`...
  = note: ...but it actually implements `Fn<(&'2 str,)>`, for some specific lifetime `'2`

could be improved by making the output closer to code people can write

  = note: `impl Fn(&'2 str)` has a specific lifetime `'2`, but must work for `for<'1> Fn<(&'1 str,)>` instead

@jackh726
Copy link
Member

@estebank I think you're correct, but I think that might be better for a separate issue?

@estebank
Copy link
Contributor

estebank commented Jun 1, 2022

@jackh726 yes, a separate PR is always in the table for these kind of fixes :)

Sorry, a separate ticket... maybe. I do tend to group things in the same ticket or hijack an earlier one when it makes sense, which I feel it does here, but I'm not against more, smaller tickets.

@jackh726
Copy link
Member

jackh726 commented Jun 1, 2022

Imo this isn't the right issue for that. I think this should be purely to remove any "bad higher ranked subtype"-like errors. As it stands, nearly all have been removed and match non-NLL behavior. A separate issue could be opened to improve that error in NLL (unlikely to be relevant for non-NLL, since it'll likely get stabilized soon).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-NLL Area: Non Lexical Lifetimes (NLL) NLL-diagnostics Working torwads the "diagnostic parity" goal P-medium Medium priority T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

10 participants