Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add -Z borrowck=migrate #52681

Merged
merged 10 commits into from Jul 27, 2018
Merged

Add -Z borrowck=migrate #52681

merged 10 commits into from Jul 27, 2018

Conversation

pnkfelix
Copy link
Member

@pnkfelix pnkfelix commented Jul 24, 2018

This adds -Z borrowck=migrate, which represents the way we want to migrate to NLL under Rust versions to come. It also hooks this new mode into --edition 2018, which means we're officially turning NLL on in the 2018 edition.

The basic idea of -Z borrowck=migrate that there are cases where NLL is fixing old soundness bugs in the borrow-checker, but in order to avoid just breaking code by immediately rejecting the programs that hit those soundness bugs, we instead use the following strategy:

If your code is accepted by NLL, then we accept it.
If your code is rejected by both NLL and the old AST-borrowck, then we reject it.
If your code is rejected by NLL but accepted by the old AST-borrowck, then we emit the new NLL errors as warnings.

These warnings will be turned into hard errors in the future, and they say so in these diagnostics.

Fix #46908

@rust-highfive
Copy link
Collaborator

r? @eddyb

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 24, 2018
@pnkfelix
Copy link
Member Author

r? @nikomatsakis

@rust-highfive rust-highfive assigned nikomatsakis and unassigned eddyb Jul 24, 2018
@nikomatsakis
Copy link
Contributor

r=me -- but @pnkfelix and I discussed on Zulip the idea of adding some tests to test the interaction of #![feature(nll)] and -Zborrowck=migrate. The expectation is roughly that #![feature(nll)] will give hard errors -- and also that --edition 2018 will switch to migrate mode.

@nikomatsakis
Copy link
Contributor

Or we might do the --edition thing later

@pnkfelix
Copy link
Member Author

@bors r=nikomatsakis

@bors
Copy link
Contributor

bors commented Jul 25, 2018

📌 Commit 189b92d93d5ebdf64aec197cddc33dcc1541e75a has been approved by nikomatsakis

@bors
Copy link
Contributor

bors commented Jul 25, 2018

🌲 The tree is currently closed for pull requests below priority 99, this pull request will be tested once the tree is reopened

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 25, 2018
Also convert an ICE that became reachable code under borrowck=migrate
into a normally reported error (which is then downgraded to a
warning). This actually has a nice side benefit of providing a
somewhat more useful error message, at least in the particular case of
the example from issue rust-lang#27282.
Note that this test is carefully crafted to *try* to not segfault
during its run.  Howver, it really is representing unsound code that
should be rejected after we manage to remove the AST-borrowck entirely
from the compiler.
(Includes test illustrating desired behavior; compare its diagnostic
output to that of the file `borrowck-migreate-to-nll.rs`.)
@pnkfelix
Copy link
Member Author

@bors r-

Since the tree is closed anyway, I'll hold off on landing this until after I have the edition linkage ready to go.

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 26, 2018
As a driveby change, I made `#![feature(nll)]` *always* take
precedence over `-Z borrowck`.  The main effect this had is that it
means tests with `#![feature(nll)]` will ignore uses of `-Z
borrowck=compare`. This affected only one test as far as I can tell,
and I think that test used `-Z borrowck=compare` only as a historical
accident.
@pnkfelix
Copy link
Member Author

r? @nikomatsakis to just double-check the commits I've added on, starting from "Bug fix: #![feature(nll)] takes precedence over -Z borrowck=migrate.

@pnkfelix pnkfelix added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 26, 2018
@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 27, 2018
@bors
Copy link
Contributor

bors commented Jul 27, 2018

⌛ Testing commit 9f05f29 with merge dfae368436e951c8583447ee673b3a912153fd6f...

@bors
Copy link
Contributor

bors commented Jul 27, 2018

💔 Test failed - status-travis

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 27, 2018
@rust-highfive
Copy link
Collaborator

The job dist-i686-freebsd of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:07812816:start=1532675755719172506,finish=1532675755726090284,duration=6917778
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:3986acd7
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:10bc68bf
travis_time:start:10bc68bf
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:087ed1e5
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@kennytm
Copy link
Member

kennytm commented Jul 27, 2018

@bors retry travis-ci/travis-ci#9696

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 27, 2018
bors added a commit that referenced this pull request Jul 27, 2018
Add `-Z borrowck=migrate`

This adds `-Z borrowck=migrate`, which represents the way we want to migrate to NLL under Rust versions to come. It also hooks this new mode into `--edition 2018`, which means we're officially turning NLL on in the 2018 edition.

The basic idea of `-Z borrowck=migrate` that there are cases where NLL is fixing old soundness bugs in the borrow-checker, but in order to avoid just breaking code by immediately rejecting the programs that hit those soundness bugs, we instead use the following strategy:

If your code is accepted by NLL, then we accept it.
If your code is rejected by both NLL and the old AST-borrowck, then we reject it.
If your code is rejected by NLL but accepted by the old AST-borrowck, then we emit the new NLL errors as **warnings**.

These warnings will be turned into hard errors in the future, and they say so in these diagnostics.

Fix #46908
@bors
Copy link
Contributor

bors commented Jul 27, 2018

⌛ Testing commit 9f05f29 with merge b18b9ed...

@RalfJung
Copy link
Member

If your code is accepted by NLL, then we accept it.

Does that mean AST borrowck will not even run unless NLL borrowck fails?

@bors
Copy link
Contributor

bors commented Jul 27, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing b18b9ed to master...

@bors bors merged commit 9f05f29 into rust-lang:master Jul 27, 2018
@pnkfelix
Copy link
Member Author

@RalfJung asked:

Does that mean AST borrowck will not even run unless NLL borrowck fails?

That's right. Or at least, that's the intent.

bors added a commit to rust-lang-ci/rust that referenced this pull request Jun 7, 2022
…matsakis

Remove migrate borrowck mode

Closes rust-lang#58781
Closes rust-lang#43234

# Stabilization proposal

This PR proposes the stabilization of `#![feature(nll)]` and the removal of `-Z borrowck`. Current borrow checking behavior of item bodies is currently done by first infering regions *lexically* and reporting any errors during HIR type checking. If there *are* any errors, then MIR borrowck (NLL) never occurs. If there *aren't* any errors, then MIR borrowck happens and any errors there would be reported. This PR removes the lexical region check of item bodies entirely and only uses MIR borrowck. Because MIR borrowck could never *not* be run for a compiled program, this should not break any programs. It does, however, change diagnostics significantly and allows a slightly larger set of programs to compile.

Tracking issue: rust-lang#43234
RFC: https://github.com/rust-lang/rfcs/blob/master/text/2094-nll.md
Version: 1.63 (2022-06-30 => beta, 2022-08-11 => stable).

## Motivation

Over time, the Rust borrow checker has become "smarter" and thus allowed more programs to compile. There have been three different implementations: AST borrowck, MIR borrowck, and polonius (well, in progress). Additionally, there is the "lexical region resolver", which (roughly) solves the constraints generated through HIR typeck. It is not a full borrow checker, but does emit some errors.

The AST borrowck was the original implementation of the borrow checker and was part of the initially stabilized Rust 1.0. In mid 2017, work began to implement the current MIR borrow checker and that effort ompleted by the end of 2017, for the most part. During 2018, efforts were made to migrate away from the AST borrow checker to the MIR borrow checker - eventually culminating into "migrate" mode - where HIR typeck with lexical region resolving following by MIR borrow checking - being active by default in the 2018 edition.

In early 2019, migrate mode was turned on by default in the 2015 edition as well, but with MIR borrowck errors emitted as warnings. By late 2019, these warnings were upgraded to full errors. This was followed by the complete removal of the AST borrow checker.

In the period since, various errors emitted by the MIR borrow checker have been improved to the point that they are mostly the same or better than those emitted by the lexical region resolver.

While there do remain some degradations in errors (tracked under the [NLL-diagnostics tag](https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3ANLL-diagnostics), those are sufficiently small and rare enough that increased flexibility of MIR borrow check-only is now a worthwhile tradeoff.

## What is stabilized

As said previously, this does not fundamentally change the landscape of accepted programs. However, there are a [few](https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3ANLL-fixed-by-NLL) cases where programs can compile under `feature(nll)`, but not otherwise.

There are two notable patterns that are "fixed" by this stabilization. First, the `scoped_threads` feature, which is a continutation of a pre-1.0 API, can sometimes emit a [weird lifetime error](rust-lang#95527) without NLL. Second, actually seen in the standard library. In the `Extend` impl for `HashMap`, there is an implied bound of `K: 'a` that is available with NLL on but not without - this is utilized in the impl.

As mentioned before, there are a large number of diagnostic differences. Most of them are better, but some are worse. None are serious or happen often enough to need to block this PR. The biggest change is the loss of error code for a number of lifetime errors in favor of more general "lifetime may not live long enough" error. While this may *seem* bad, the former error codes were just attempts to somewhat-arbitrarily bin together lifetime errors of the same type; however, on paper, they end up being roughly the same with roughly the same kinds of solutions.

## What isn't stabilized

This PR does not completely remove the lexical region resolver. In the future, it may be possible to remove that (while still keeping HIR typeck) or to remove it together with HIR typeck.

## Tests

Many test outputs get updated by this PR. However, there are number of tests specifically geared towards NLL under `src/test/ui/nll`

## History

* On 2017-07-14, [tracking issue opened](rust-lang#43234)
* On 2017-07-20, [initial empty MIR pass added](rust-lang#43271)
* On 2017-08-29, [RFC opened](rust-lang/rfcs#2094)
* On 2017-11-16, [Integrate MIR type-checker with NLL](rust-lang#45825)
* On 2017-12-20, [NLL feature complete](rust-lang#46862)
* On 2018-07-07, [Don't run AST borrowck on mir mode](rust-lang#52083)
* On 2018-07-27, [Add migrate mode](rust-lang#52681)
* On 2019-04-22, [Enable migrate mode on 2015 edition](rust-lang#59114)
* On 2019-08-26, [Don't downgrade errors on 2015 edition](rust-lang#64221)
* On 2019-08-27, [Remove AST borrowck](rust-lang#64790)
flip1995 pushed a commit to flip1995/rust-clippy that referenced this pull request Jun 16, 2022
Remove migrate borrowck mode

Closes #58781
Closes #43234

# Stabilization proposal

This PR proposes the stabilization of `#![feature(nll)]` and the removal of `-Z borrowck`. Current borrow checking behavior of item bodies is currently done by first infering regions *lexically* and reporting any errors during HIR type checking. If there *are* any errors, then MIR borrowck (NLL) never occurs. If there *aren't* any errors, then MIR borrowck happens and any errors there would be reported. This PR removes the lexical region check of item bodies entirely and only uses MIR borrowck. Because MIR borrowck could never *not* be run for a compiled program, this should not break any programs. It does, however, change diagnostics significantly and allows a slightly larger set of programs to compile.

Tracking issue: #43234
RFC: https://github.com/rust-lang/rfcs/blob/master/text/2094-nll.md
Version: 1.63 (2022-06-30 => beta, 2022-08-11 => stable).

## Motivation

Over time, the Rust borrow checker has become "smarter" and thus allowed more programs to compile. There have been three different implementations: AST borrowck, MIR borrowck, and polonius (well, in progress). Additionally, there is the "lexical region resolver", which (roughly) solves the constraints generated through HIR typeck. It is not a full borrow checker, but does emit some errors.

The AST borrowck was the original implementation of the borrow checker and was part of the initially stabilized Rust 1.0. In mid 2017, work began to implement the current MIR borrow checker and that effort ompleted by the end of 2017, for the most part. During 2018, efforts were made to migrate away from the AST borrow checker to the MIR borrow checker - eventually culminating into "migrate" mode - where HIR typeck with lexical region resolving following by MIR borrow checking - being active by default in the 2018 edition.

In early 2019, migrate mode was turned on by default in the 2015 edition as well, but with MIR borrowck errors emitted as warnings. By late 2019, these warnings were upgraded to full errors. This was followed by the complete removal of the AST borrow checker.

In the period since, various errors emitted by the MIR borrow checker have been improved to the point that they are mostly the same or better than those emitted by the lexical region resolver.

While there do remain some degradations in errors (tracked under the [NLL-diagnostics tag](https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3ANLL-diagnostics), those are sufficiently small and rare enough that increased flexibility of MIR borrow check-only is now a worthwhile tradeoff.

## What is stabilized

As said previously, this does not fundamentally change the landscape of accepted programs. However, there are a [few](https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3ANLL-fixed-by-NLL) cases where programs can compile under `feature(nll)`, but not otherwise.

There are two notable patterns that are "fixed" by this stabilization. First, the `scoped_threads` feature, which is a continutation of a pre-1.0 API, can sometimes emit a [weird lifetime error](rust-lang/rust#95527) without NLL. Second, actually seen in the standard library. In the `Extend` impl for `HashMap`, there is an implied bound of `K: 'a` that is available with NLL on but not without - this is utilized in the impl.

As mentioned before, there are a large number of diagnostic differences. Most of them are better, but some are worse. None are serious or happen often enough to need to block this PR. The biggest change is the loss of error code for a number of lifetime errors in favor of more general "lifetime may not live long enough" error. While this may *seem* bad, the former error codes were just attempts to somewhat-arbitrarily bin together lifetime errors of the same type; however, on paper, they end up being roughly the same with roughly the same kinds of solutions.

## What isn't stabilized

This PR does not completely remove the lexical region resolver. In the future, it may be possible to remove that (while still keeping HIR typeck) or to remove it together with HIR typeck.

## Tests

Many test outputs get updated by this PR. However, there are number of tests specifically geared towards NLL under `src/test/ui/nll`

## History

* On 2017-07-14, [tracking issue opened](rust-lang/rust#43234)
* On 2017-07-20, [initial empty MIR pass added](rust-lang/rust#43271)
* On 2017-08-29, [RFC opened](rust-lang/rfcs#2094)
* On 2017-11-16, [Integrate MIR type-checker with NLL](rust-lang/rust#45825)
* On 2017-12-20, [NLL feature complete](rust-lang/rust#46862)
* On 2018-07-07, [Don't run AST borrowck on mir mode](rust-lang/rust#52083)
* On 2018-07-27, [Add migrate mode](rust-lang/rust#52681)
* On 2019-04-22, [Enable migrate mode on 2015 edition](rust-lang/rust#59114)
* On 2019-08-26, [Don't downgrade errors on 2015 edition](rust-lang/rust#64221)
* On 2019-08-27, [Remove AST borrowck](rust-lang/rust#64790)
workingjubilee pushed a commit to tcdi/postgrestd that referenced this pull request Sep 15, 2022
Remove migrate borrowck mode

Closes #58781
Closes #43234

# Stabilization proposal

This PR proposes the stabilization of `#![feature(nll)]` and the removal of `-Z borrowck`. Current borrow checking behavior of item bodies is currently done by first infering regions *lexically* and reporting any errors during HIR type checking. If there *are* any errors, then MIR borrowck (NLL) never occurs. If there *aren't* any errors, then MIR borrowck happens and any errors there would be reported. This PR removes the lexical region check of item bodies entirely and only uses MIR borrowck. Because MIR borrowck could never *not* be run for a compiled program, this should not break any programs. It does, however, change diagnostics significantly and allows a slightly larger set of programs to compile.

Tracking issue: #43234
RFC: https://github.com/rust-lang/rfcs/blob/master/text/2094-nll.md
Version: 1.63 (2022-06-30 => beta, 2022-08-11 => stable).

## Motivation

Over time, the Rust borrow checker has become "smarter" and thus allowed more programs to compile. There have been three different implementations: AST borrowck, MIR borrowck, and polonius (well, in progress). Additionally, there is the "lexical region resolver", which (roughly) solves the constraints generated through HIR typeck. It is not a full borrow checker, but does emit some errors.

The AST borrowck was the original implementation of the borrow checker and was part of the initially stabilized Rust 1.0. In mid 2017, work began to implement the current MIR borrow checker and that effort ompleted by the end of 2017, for the most part. During 2018, efforts were made to migrate away from the AST borrow checker to the MIR borrow checker - eventually culminating into "migrate" mode - where HIR typeck with lexical region resolving following by MIR borrow checking - being active by default in the 2018 edition.

In early 2019, migrate mode was turned on by default in the 2015 edition as well, but with MIR borrowck errors emitted as warnings. By late 2019, these warnings were upgraded to full errors. This was followed by the complete removal of the AST borrow checker.

In the period since, various errors emitted by the MIR borrow checker have been improved to the point that they are mostly the same or better than those emitted by the lexical region resolver.

While there do remain some degradations in errors (tracked under the [NLL-diagnostics tag](https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3ANLL-diagnostics), those are sufficiently small and rare enough that increased flexibility of MIR borrow check-only is now a worthwhile tradeoff.

## What is stabilized

As said previously, this does not fundamentally change the landscape of accepted programs. However, there are a [few](https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3ANLL-fixed-by-NLL) cases where programs can compile under `feature(nll)`, but not otherwise.

There are two notable patterns that are "fixed" by this stabilization. First, the `scoped_threads` feature, which is a continutation of a pre-1.0 API, can sometimes emit a [weird lifetime error](rust-lang/rust#95527) without NLL. Second, actually seen in the standard library. In the `Extend` impl for `HashMap`, there is an implied bound of `K: 'a` that is available with NLL on but not without - this is utilized in the impl.

As mentioned before, there are a large number of diagnostic differences. Most of them are better, but some are worse. None are serious or happen often enough to need to block this PR. The biggest change is the loss of error code for a number of lifetime errors in favor of more general "lifetime may not live long enough" error. While this may *seem* bad, the former error codes were just attempts to somewhat-arbitrarily bin together lifetime errors of the same type; however, on paper, they end up being roughly the same with roughly the same kinds of solutions.

## What isn't stabilized

This PR does not completely remove the lexical region resolver. In the future, it may be possible to remove that (while still keeping HIR typeck) or to remove it together with HIR typeck.

## Tests

Many test outputs get updated by this PR. However, there are number of tests specifically geared towards NLL under `src/test/ui/nll`

## History

* On 2017-07-14, [tracking issue opened](rust-lang/rust#43234)
* On 2017-07-20, [initial empty MIR pass added](rust-lang/rust#43271)
* On 2017-08-29, [RFC opened](rust-lang/rfcs#2094)
* On 2017-11-16, [Integrate MIR type-checker with NLL](rust-lang/rust#45825)
* On 2017-12-20, [NLL feature complete](rust-lang/rust#46862)
* On 2018-07-07, [Don't run AST borrowck on mir mode](rust-lang/rust#52083)
* On 2018-07-27, [Add migrate mode](rust-lang/rust#52681)
* On 2019-04-22, [Enable migrate mode on 2015 edition](rust-lang/rust#59114)
* On 2019-08-26, [Don't downgrade errors on 2015 edition](rust-lang/rust#64221)
* On 2019-08-27, [Remove AST borrowck](rust-lang/rust#64790)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Want -Z borrowck=migrate to test planning migration to non-lexical lifetimes
7 participants