VecDeque: fix for stacked borrows

[RalfJung]

VecDeque violates a version of stacked borrows where creating a shared reference is not enough to make a location mutably accessible from raw pointers (and I think that is the version we want). There are two problems:

• Creating a NonNull<T> from &mut T goes through &T (inferred for a _), then *const T, then NonNull<T>. That means in this stricter version of Stacked Borrows, we cannot actually write to such a NonNull because it was created from a shared reference! This PR fixes that by going from &mut T to *mut T to *const T.
• VecDeque::drain creates the Drain struct by first creating a NonNull from self (which is an &mut VecDeque), and then calling self.buffer_as_mut_slice(). The latter reborrows self, asserting that self is currently the unique pointer to access this VecDeque, and hence invalidating the NonNull that was created earlier. This PR fixes that by instead using self.buffer_as_slice(), which only performs read accesses and creates only shared references, meaning the raw pointer (NonNull) remains valid.

It is possible that other methods on VecDeque do something similar, miri's test coverage of VecDeque is sparse to say the least.

Cc @nikomatsakis @Gankro

[rust-highfive]

r? @KodrAus

(rust_highfive has picked a reviewer for you, use r? to override)

[RalfJung]

Turns out just not using a mutable buffer, as suggested by @gnzlbg, is enough :)

[gnzlbg]

LGTM.

[gnzlbg]

LGTM.

[bluss]

Since this is super tricky stuff, it would be awesome if the PR description was updated to correspond to the final state of the PR .

[RalfJung]

@bluss Good point, done.

[bluss]

If I understand you correctly, the _ here was filled in with &'a T, and that's no good?

[RalfJung]

Yes.

Ideally we could make it infer *mut T for the _, that would be the safer choice.

[RalfJung]

Rebased (because I want to test this together with #56165)

[bors]

☔️ The latest upstream changes (presumably #55635) made this pull request unmergeable. Please resolve the merge conflicts.

[RalfJung]

This PR is now awaiting review for 12 days.

Ping @KodrAus

[gnzlbg]

r? me

[RalfJung]

r? @gnzlbg

[SimonSapin]

These are really subtle rules, and I’m worried about what this means for unsafe code outside of the standard library that might similarly violate those rules.

This combination feels very dangerous:

• Going from &mut T to &T is a sort of "assertion of immutability" that has very significant meaning for the soundness of unsafe code
• This coercion can happen implicitly without as
• It even happens in the middle of foo as _ where foo: &mut T in a context that expects *const T, without the &T type being named anywhere

[SimonSapin]

I think I feel strongly enough about this that I’m opposed to landing this PR as-is. Making soundness rely on the non-existence of a shared reference is not acceptable IMO as long as such a shared reference can be created "invisibly" like this.

[gnzlbg]

@SimonSapin I am not sure that I understand your argument correctly. Are you arguing that it is easy to assert immutability accidentally and therefore introduce undefined behavior ?

If so, I agree. But this code was doing so explicitly with as _ where a lot was going on independently of whether this PR is merged or not. This PR is more explicit and clear (e.g. as *mut T instead of as _) and undefined behavior free (EDIT: at least, with the current stacked borrows semantics).

There are a couple of ideas being floated around about how to prevent "asserting immutability by accident" (e.g. making as/coercion chains "atomic"), and one of these ideas already has an RFC (rust-lang/rfcs#2582). But more RFCs will be needed to at least reduce the degree by which this can happen by accident.

[SimonSapin]

Yes, this is what I’m arguing.

We have here a combination of three things:

1. The in the newly-proposed stacked borrows model, mutating through a raw pointer that was crated from a shared reference is not allowed.
2. Converting from &mut T to *const T with x as _ silently goes through &T.
3. We found one instance of 2. happening with mutation, violating the rules of 1.

It’s only with all three combined that there’s a problem. So we can fix the problem in any of three ways:

A. Not accepting this proposed model
B. Changing as so it doesn’t do that
C. Fixing the code in this one instance

I am opposed to doing only C. because that doen’t help with other potential instances of the same problem. This is made worse by 2. being, in my opinion, very unexpected. I claim to be fairly proficient at Rust, and I had no idea this was happening. x as *const T is also allowed, why would _ not be inferred to *const T in this case? I suspect this was only discovered through mechanical checking by miri, but it’s not practical to declare that all unsafe Rust code in the world must be checked by miri. (For example, that is not possible when FFI is involved.)

So I think we should fix as and not land (the ptr.rs part of) this PR.

[RalfJung]

Going from &mut T to &T is a sort of "assertion of immutability" that has very significant meaning for the soundness of unsafe code

From my understanding, this was an explicit design goal from the start. Cc @eddyb @Gankro @nikomatsakis

Not doing this kills a lot of useful optimizations, because it means that creating a shared reference gives license to anyone to mutate data (until a barrier is added). In particular, the change I am making here is a necessary requirement to ever solve #16515 for shared references.

mutating through a raw pointer that was crated from a shared reference is not allowed.

This was one of the first rules I ever learned about mutability in Rust: Though shall not mutate through shared references. Of course this has to include raw pointers created from shared references, otherwise the rule is useless (the compiler syntactically prevents direct assignment to a shared reference). This rule is the reason we have UnsafeCell. It is fundamental, and it is ancient.

It even happens in the middle of foo as _ where foo: &mut T in a context that expects *const T, without the &T type being named anywhere

That, I think, is a problem. We should change the coercion logic to only coerce to &T if needed.

---

The reason I would like to land this PR is that it is needed to move forward with experimenting with this model in miri. I do not see this as acknowledgement that these will ultimately be the rules. But as long as a core data structure like NonNull breaks them, we cannot even reasonably use miri to see which effect this stricter rule has in the wild.

When I previously talked about this with @nikomatsakis, he was open to changing the coercion rules in principle, but also said that coercion is a mess and in its current state maybe should not be touched, but rewritten. This PR is a way forward for experimentation with miri that is not blocked on fixing coercion. But I absolutely agree that we should block accepting this model and exploiting it in the compiler on fixing coercions. I am not sure what would be the best place to track this.

[eddyb]

So is the problem here that x as _ infers the _ to &T instead of *const T?
If so, I agree it needs to be fixed. I think we shouldn't coerce the result of a cast if we can infer the cast to that coercion. We're only forced to do an extra coercion if the cast type is more explicit.

but also said that coercion is a mess and in its current state maybe should not be touched, but rewritten

I hope/suspect "rewritten" here doesn't mean just rewriting the librustc_typeck/check/coerce.rs file, since that wouldn't do much good, but rather integrating coercions with the trait system.

[gnzlbg]

So is the problem here that x as _ infers the _ to &T instead of *const T?

The problem is that coercion chains are not atomic. x as _ does infers the _ to a &T, which is arguably wrong, but note that the as _ is not necessary here, x is a reference, and it can coerce to a raw pointer. This does still fail, because the coercion still makes it go through a reference.

I have no idea what as _ was trying to achieve here, but it doesn't really do anything. That is, this code needs changing no matter what (either to x as *mut T or just to x).

[gnzlbg]

FWIW, I'm fine to changing this to x as *mut T for now, but in the future that should raise a warning because the as *mut T (or as _) are unnecessary.

[eddyb]

Are let y: *mut T = x; and x as *mut T different, at all? That sounds very wrong.

[RalfJung]

(I wish we had kept a single *T type without the not-so-useful *const T v.s. *mut T separation…)

Same here, except I didn't even know that it used to be a single type. Alas, too late.

---

@eddyb I think the relevant code is

rust/src/librustc_typeck/check/coercion.rs

Lines 734 to 752 in f504d3f

	coerce_mutbls(mt_a.mutbl, mutbl_b)?;
	// Although references and unsafe ptrs have the same
	// representation, we still register an Adjust::DerefRef so that
	// regionck knows that the region for `a` must be valid here.
	if is_ref {
	self.unify_and(a_unsafe, b, |target| {
	vec![Adjustment {
	kind: Adjust::Deref(None),
	target: mt_a.ty
	}, Adjustment {
	kind: Adjust::Borrow(AutoBorrow::RawPtr(mutbl_b)),
	target
	}]
	})
	} else if mt_a.mutbl != mutbl_b {
	self.unify_and(a_unsafe, b, simple(Adjust::MutToConstPointer))
	} else {
	self.unify_and(a_unsafe, b, identity)
	}

any hint what I'd have to change there to make it not create a shared reference, but instead two casts? Or do we just want to allow direct cast from &mut T to *const T?

[eddyb]

Or do we just want to allow direct cast from &mut T to *const T?

That's what we want, yes.

[RalfJung]

I opened #56604 for the coercion problem, let's continue the discussion there.

[RalfJung]

@SimonSapin Beyond the issue above, I have added a not-yet-working test with a FIXME to miri and recorded this in the "missing validation tracking issue" at rust-lang/miri#551. Is that enough to remedy your concerns about this change?

libtest uses BTreeMap uses NonNull, so making cargo miri test work at all needs some kind of solution to this problem. I have no idea how long it will take to sort out the coercion problem, but I'm afraid it might take a while. That's why I'd like to land this.

I could change miri to be less strict about the "writing through raw pointers obtained from shared references" rule, but it seems strange to accept code that we definitely know we do not want to accept.

[RalfJung]

libtest uses BTreeMap uses NonNull, so making cargo miri test work at all needs some kind of solution to this problem.

Turns out that BTreeMap does not use the problematic parts of NonNull, so this PR doesn't actually block getting cargo miri test to work.

Still, seems likely that other code (besides VecDeque) will also run into these "bad" conversion functions.

[SimonSapin]

The reason I would like to land this PR is that it is needed to move forward with experimenting with this model in miri.

Ok, landing this PR to unblock other work sounds acceptable, with the understanding that it is not "the" way to fix this kind of unsoundess (#56604 is) and that unsafe code outside of std does not need to be audited for this pattern.

[Centril]

@SimonSapin As a process note (not an opinion on #56604 or rust-lang/rfcs#2582), such an understanding depends on consensus within T-lang; Since this is a soundness fix, we should just land it with a FIXME(RalfJung) or something like that and revisit if and when rules change. Blocking it on anything else seems like linking things together that shouldn't be linked.

[Gankro]

It’s not actually a soundness fix since nothing in codegen relies on the model this code violates

[RalfJung]

the understanding that it is not "the" way to fix this kind of unsoundess (#56604 is) and that unsafe code outside of std does not need to be audited for this pattern.

Agreed.

Okay so can I get someone to r+ this? :D

[SimonSapin]

@bors r+

[bors]

📌 Commit feb775c has been approved by SimonSapin

[bors]

⌛️ Testing commit feb775c with merge 9fe5cb5...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: SimonSapin
Pushing 9fe5cb5 to master...

[rust-highfive]

📣 Toolstate changed by #56161!

Tested on commit 9fe5cb5.
Direct link to PR: #56161

💔 rls on linux: test-pass → test-fail (cc @nrc @Xanewok, @rust-lang/infra).

[Xanewok]

@alexcrichton @kennytm this is intermittent - any possibility to retrigger the -tools CI job and change current toolstate without submitting a new PR?

(On a similar note, why do newest nightlies don't contain RLS or rustfmt? The toolstate was ok and the manifest tool was fixed from what I've seen - anything else blocking these from distribution?)

[RalfJung]

why do newest nightlies don't contain RLS or rustfmt? The toolstate was ok and the manifest tool was fixed from what I've seen - anything else blocking these from distribution?

That would be #56667. There simply wasn't a new nightly since the tools were fixed.

[Xanewok]

Ah, so it's not because the nightlies are blocked on broken tools but there's a problem with distributing nightlies itself! Makes sense, thanks for the heads up.

EDIT: Or maybe not (it seems clippy managed to break just before the distribution). In any case, hopefully the next nightlies are okay.