Make closure capturing have consistent and correct behaviour around patterns

[meithecatte]

This PR has two goals:

• firstly, it fixes internal compiler error: two identical projections #137467. In order to do so, it needs to introduce a small breaking change surrounding the interaction of closure captures with matching against enums with uninhabited variants. Yes – to fix an ICE!
  • this also fixes ICE: called Option::unwrap() on a None value with refutable patterns #138973, a slightly different case with the same root cause.
• secondly, it fixes Closure captures are inconsistent between x and x @ _ irrefutable patterns #137553, making the closure capturing rules consistent between let patterns and match patterns. This is new insta-stable behavior.

Background

This change concerns how precise closure captures interact with patterns. As a little known feature, patterns that require inspecting only part of a value will only cause that part of the value to get captured:

fn main() {
    let mut a = (21, 37);
    // only captures a.0, writing to a.1 does not invalidate the closure
    let mut f = || {
        let (ref mut x, _) = a;
        *x = 42;
    };
    a.1 = 69;
    f();
}

I was not able to find any discussion of this behavior being introduced, or discussion of its edge-cases, but it is documented in the Rust reference.

The currently stable behavior is as follows:

• if any pattern contains a binding, the place it binds gets captured (implemented in current walk_pat)
• patterns in refutable positions (match, if let, let ... else, but not destructuring let or destructuring function parameters) get processed as follows (maybe_read_scrutinee):
  • if matching against the pattern will at any point require inspecting a discriminant, or it includes a variable binding not followed by an @-pattern, capture the entire scrutinee by reference

You will note that this behavior is quite weird and it's hard to imagine a sensible rationale for at least some of its aspects. It has the following issues:

• firstly, it assumes that matching against an irrefutable pattern cannot possibly require inspecting any discriminants. With or-patterns, this isn't true, and it is the cause of the internal compiler error: two identical projections #137467 ICE.
• secondly, the presence of an @-pattern doesn't really have any semantics by itself. This is the weird behavior tracked as Closure captures are inconsistent between x and x @ _ irrefutable patterns #137553.
• thirdly, the behavior is different between pattern-matching done through let and pattern-matching done through match – which is a superficial syntactic difference

This PR aims to address all of the above issues. The new behavior is as follows:

• like before, if a pattern contains a binding, the place it binds gets captured as required by the binding mode
• if matching against the pattern requires inspecting a disciminant, the place whose discriminant needs to be inspected gets captured by reference

"requires inspecting a discriminant" is also used here to mean "compare something with a constant" and other such decisions. For types other than ADTs, the details are not interesting and aren't changing.

The breaking change

During closure capture analysis, matching an enum against a constructor is considered to require inspecting a discriminant if the enum has more than one variant. Notably, this is the case even if all the other variants happen to be uninhabited. This is motivated by implementation difficulties involved in querying whether types are inhabited before we're done with type inference – without moving mountains to make it happen, you hit this assert:

rust/compiler/rustc_middle/src/ty/inhabitedness/mod.rs

Line 121 in 43f0014

debug_assert!(!self.has_infer());

Now, because the previous implementation did not concern itself with capturing the discriminants for irrefutable patterns at all, this is a breaking change – the following example, adapted from the testsuite, compiles on current stable, but will not compile with this PR:

#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum Void {}

pub fn main() {
    let mut r = Result::<Void, (u32, u32)>::Err((0, 0));
    let mut f = || {
        let Err((ref mut a, _)) = r;
        *a = 1;
    };
    let mut g = || {
    //~^ ERROR: cannot borrow `r` as mutable more than once at a time
        let Err((_, ref mut b)) = r;
        *b = 2;
    };
    f();
    g();
    assert_eq!(r, Err((1, 2)));
}

Is the breaking change necessary?

One other option would be to double down, and introduce a set of syntactic rules for determining whether a sub-pattern is in an irrefutable position, instead of querying the types and checking how many variants there are.

This would not eliminate the breaking change, but it would limit it to more contrived examples, such as

let ((true, Err((ref mut a, _, _))) | (false, Err((_, ref mut a, _)))) = x;

In this example, the Errs would not be considered in an irrefutable position, because they are part of an or-pattern. However, current stable would treat this just like a tuple (bool, (T, U, _)).

While introducing such a distinction would limit the impact, I would say that the added complexity would not be commensurate with the benefit it introduces.

The new insta-stable behavior

If a pattern in a match expression or similar has parts it will never read, this part will not be captured anymore:

fn main() {
    let mut a = (21, 37);
    // now only captures a.0, instead of the whole a
    let mut f = || {
        match a {
            (ref mut x, _) => *x = 42,
        }
    };
    a.1 = 69;
    f();
}

Note that this behavior was pretty much already present, but only accessible with this One Weird Trick™:

fn main() {
    let mut a = (21, 37);
    // both stable and this PR only capture a.0, because of the no-op @-pattern
    let mut f = || {
        match a {
            (ref mut x @ _, _) => *x = 42,
        }
    };
    a.1 = 69;
    f();
}

Implementation notes

The first commits of the PR perform various cleanups. The action happens in two parts:

• "ExprUseVisitor: properly report discriminant reads" makes walk_pat perform all necessary capturing. This is the part that fixes internal compiler error: two identical projections #137467.
• "ExprUseVisitor: remove maybe_read_scrutinee" removes the unnecessary "capture the entire scrutinee" behavior, fixing Closure captures are inconsistent between x and x @ _ irrefutable patterns #137553.

The new logic stops making the distinction between one particular example that used to work, and another ICE, tracked as #119786. As this requires an unstable feature, I am leaving this as future work.

[rustbot]

This PR changes a file inside tests/crashes. If a crash was fixed, please move into the corresponding ui subdir and add 'Fixes #' to the PR description to autoclose the issue upon merge.

[meithecatte]

Nadrieril suggested that this should be resolved through a breaking change – updated the PR description accordingly.

@rustbot label +needs-crater

r? @Nadrieril

[rustbot]

Error: Label needs-crater can only be set by Rust team members

Please file an issue on GitHub at triagebot if there's a problem with this bot, or reach out on #t-infra on Zulip.

[meithecatte]

@compiler-errors You've requested that the fix for #137553 land in a separate PR. However, ironically, the breaking changes are actually required by #137467 and not #137553. Do you think the removal of the now-obsolete maybe_read_scrutinee should happen in a separate PR, or should I do it here so that it also benefits from the crater run?

[compiler-errors]

We can crater both together if you think they're not worth separating. I was just trying to accelerate landing the parts that are obviously-not-breaking but it's up to you if you think that effort is worth it or if you're willing to be patient about waiting for the breaking parts (and FCP, etc).

@bors try

[bors]

⌛ Trying commit 8ed61e4 with merge 3b30da3...

[meithecatte]

We can crater both together if you think they're not worth separating. I was just trying to accelerate landing the parts that are obviously-not-breaking but it's up to you if you think that effort is worth it or if you're willing to be patient about waiting for the breaking parts (and FCP, etc).

That's the thing – one part is a breaking change, the other introduces insta-stable new behavior. There's no easily mergeable part to this.

[compiler-errors]

could we give this a less weird pr title pls 💀

@bors try

[compiler-errors]

i'll manually invoke crater since it won't be able to pick up that the existing try build (https://github.com/rust-lang-ci/rust/actions/runs/14092448811/job/39472281414) is still running.

[meithecatte]

@compiler-errors the build has finished btw

[compiler-errors]

@craterbot run mode=check-only start=master#19cab6b878ab18dce4816d85ac52b317214c485f end=try#630b4e873619ba7f396880337285be042468cd7f

[craterbot]

👌 Experiment pr-138961 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment pr-138961 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚨 Report generation of pr-138961 failed: timed out waiting for connection
🛠️ If the error is fixed use the retry-report command.

🆘 Can someone from the infra team check in on this? @rust-lang/infra
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[compiler-errors]

@craterbot retry-report

[craterbot]

🛠️ Generation of the report for pr-138961 queued again.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment pr-138961 is completed!
📊 5 regressed and 5 fixed (604269 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[meithecatte]

Looking at the crater failures, we have random C code failing to compile, some kind of chess program that already ICEs on current nightly but OOMed on the last crater run, and an instance where matches capturing less actually introduce a breaking change:

// minimized example
enum Enum {
    Variant {
        a: u32,
        b: String,
    }
}

fn f(x: Enum) -> impl FnOnce() {
    || {
        match x {
            Enum::Variant { a, b } => {},
        }
    }
}

this used to capture borrow_imm(x), copy(x.a), consume(x.b), which got merged by compute_min_captures into consume(x).

but now it only does copy(x.a) and consume(x.b), and since copy(x.a) is treated as borrow_imm(x.a), this breaks the lifetimes.

I would argue that this counts as a bugfix: it only captures the entire thing by move because the compiler decided that the bindings actually constitute a discriminant read.

However, this does suggest that we might want to split the "remove maybe_read_scrutinee entirely" part into a separate PR... thoughts?

Also, here's a link to the Zulip thread where we've been discussing this PR: #t-compiler/help > upvar inference for patterns and InhabitedPredicate

[Nadrieril]

Dear @rust-lang/lang, this PR proposes a breaking change to the language to fix some bizarre edge cases around precise closure captures. Crater found a single breakage, in a GitHub project. WDYT?

[traviscross]

If we were to OK this, we'd end up asking for a PR to be made to the affected project, particularly as it seems to be maintained. Probably you'll want to go ahead and do this now in any case, as it will make the story a bit simpler when we pick this up.

[traviscross]

Fortunately also, the regression is in the main.rs file of a bin crate, so there won't be any dependent crates affected.

[lino-levan]

PR merged on our end. Thanks for letting us know.