Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
repr(packed) allows invalid unaligned loads #27060
Comments
huonw
added
I-crash
I-nominated
T-lang
T-compiler
labels
Jul 16, 2015
|
I... honestly thought that was expected behaviour. What can possibly be done about this is general, other than making references into packed fields a different type (or forbidding them)? |
|
This is UB, so it's clearly not expected... but yes. it's more of a design flaw than an implementation issue. Note that it's possible to make code like this misbehave even without SIMD, although it's a bit trickier; LLVM performs optimizations based on the alignment of loads. |
|
Yeah, I only used SIMD because it was the simplest way to demonstrate the problem on x86. I believe platforms like ARM are generally stricter about load alignments, so even, say, |
|
Is there a way to tell LLVM that the value could be unaligned, and so LLVM should emit code that tries to read it in a safe way for the given architecture, even if it results in slower code? |
|
@retep998: That's exactly what LLVM should have done for a packed struct. This looks like a LLVM codegen bug to me. |
|
@vadimcn this is entirely our (or rather: my) fault. We used to not emit alignment data at all, which caused misaligned accesses for small aggregates (see #23431). But now we unconditionally emit alignment data purely based on the type that we're loading, ignoring where that value is stored. In fact at the point where we create the load, we currently don't even know where that pointer comes from and can't properly handle packed structs. |
|
Perhaps we need some sort of alignment attribute that we can attach to pointers? |
|
In general we have no idea where a reference comes from, e.g. |
|
Can we make creation of references into packed structs unsafe? |
|
triage: P-high |
rust-highfive
added
P-high
and removed
I-nominated
labels
Jul 23, 2015
nikomatsakis
assigned
huonw
Jul 23, 2015
|
Seems clear there's a bug here. First step is to evaluate how widely used packed is, so huon is going to do a crater run with |
added a commit
to huonw/rust
that referenced
this issue
Jul 23, 2015
added a commit
to huonw/rust
that referenced
this issue
Jul 24, 2015
added a commit
to huonw/rust
that referenced
this issue
Jul 24, 2015
added a commit
to huonw/rust
that referenced
this issue
Jul 24, 2015
added a commit
to huonw/image
that referenced
this issue
Jul 24, 2015
added a commit
to huonw/X11Cap
that referenced
this issue
Jul 24, 2015
added a commit
to huonw/image
that referenced
this issue
Jul 24, 2015
huonw
referenced this issue
in PistonDevelopers/image
Jul 24, 2015
Merged
Remove unnecessary use of `#[repr(packed)]`. #432
added a commit
to huonw/stemmer-rs
that referenced
this issue
Jul 24, 2015
huonw
referenced this issue
in lise-henry/stemmer-rs
Jul 24, 2015
Merged
Remove unnecessary use of `#[repr(packed)]`. #1
added a commit
to huonw/rust-openal
that referenced
this issue
Jul 24, 2015
This was referenced Jul 24, 2015
added a commit
to huonw/kiss3d
that referenced
this issue
Jul 24, 2015
huonw
referenced this issue
in sebcrozet/kiss3d
Jul 24, 2015
Merged
Remove unnecessary use of `#[repr(packed)]`. #53
added a commit
to meh/rust-openal
that referenced
this issue
Jul 24, 2015
|
I made a patch that feature gated repr(packed) and did a crater run. Results: https://gist.github.com/huonw/8262a4fb2da0a052f5f1 . Summary: High-level summary:
As you can see from the "huon references this issue ..." spam above this comment, I've opened PRs against the libraries where the |
|
The bug here is the testcase, it's intentionally violating CPU alignment constraints. In C, |
|
I like @cmr's suggestion, on which I see 3 variations:
The first two have the advantage that we could pack things tighter, e.g. @huonw Could those crates be checked with an error being emitted here if |
|
cc rust-lang/rfcs#311 (and also |
|
brief sketch of how all of these could work:
(sorry for brevity, been wanting to write this down for a while but short of time) |
|
@glaebhoerl That's an interesting suggestion, and I can't think of any issues with the |
|
Oh, hmm. To be honest I hadn't even thought of that. (In case it's not clear to anyone else (it only became clear to me after I had written out a comment asking for clarification): if I have a |
|
@glaebhoerl I do expect that such a scheme would allow most code to continue compiling, given the small number of @huonw have you grepped for |
|
I use and need |
Of course the bug is in the test case... that's why I filed an issue with it ;) It's a compiler bug that the test case compiles and then crashes: it should either not compile, or not crash. It's triggering undefined behaviour (at the very least, we're telling LLVM that the load is aligned when it isn't), and, notably, it's particularly egregious, since there's no explicit internal references at all: our current (broken) scheme could probably be tweaked to make that specific example compile, without making any other changes. (Of course, the fundamental problem won't be fixed by making a few tweaks to how we codegen struct field accesses.)
Could you go into more detail about what the struct looks like? |
Nothing new. |
|
It looks like that might be able to get away with (I assume you've thought about the endianness issues that comes with mmaping data directly?) |
|
Going to throw in my voice in that Note that borrows are only UB/unsafe for types that have an alignment > 1 - I would hope to retain this behaviour at the very least, so that my Le, u8, and other such types can continue to function properly in stable and safe code. My proposal: If |
|
Not that it really matters due to the LLVM UB issue, but it's still possible to disable unaligned accesses on ARM with a CPU flag; GCC and Clang expose |
|
Can this cause memory unsafety? If it always leads to an exception and abort, maybe it is something that Rust can allow. |
|
@bluss it can. Look at the linked issue about unaligned reads on ARM, it'll just silently give you wrong/unexpected values when reading from a pointer. It's also explicitly stated as undefined behaviour in LLVM:
|
|
Well, I put an initial implementation of this into http://arcnmx.github.io/nue/packed/index.html
|
|
I've written rust-lang/rfcs#1240 which proposes making taking a reference into a |
huonw
referenced this issue
in rust-lang/rfcs
Aug 7, 2015
Merged
References into repr(packed) structs should be `unsafe`. #1240
|
I don't feel like this is the correct approach to take, will write up some notes in the RFC comments in a bit. |
|
Also, I'd like to point out that, as it turns out, a bunch of types in the Windows SDK are packed, so |
Might also be worth noting that some Windows API structs are packed to different alignments, e.g. |
|
Could we perhaps expand the syntax to |
nikomatsakis
added
the
B-RFC-approved
label
Sep 18, 2015
|
Note that rust-lang/rfcs#1240 was accepted. I have re-used this as the tracking issue. |
|
We discussed this in the @rust-lang/compiler meeting today. Nobody has written up any instructions =) but we did discuss some of the more subtle points: Currently, we strip dead code out of MIR very early. This would mean that if we do 'unsafe' checking on MIR, we will ignore dead code (e.g., There is another option: we can run the "unsafe check" before we strip dead code. The tricky part is that in that case some of the types in MIR may not match, since the type-checker ignores data-flow out of dead-code. This isn't really a problem for the unsafeck, but more generally it would mean the MIR is not "well-typed" and it'd be sort of nice to have an invariant that MIR is always well-typed. One last point: we have to be careful that |
Especially because that conditional might look like |
aturon
removed
the
I-nominated
label
Feb 23, 2017
This was referenced Mar 8, 2017
added a commit
to arielb1/rust
that referenced
this issue
Mar 9, 2017
added a commit
to arielb1/rust
that referenced
this issue
Mar 10, 2017
added a commit
to arielb1/rust
that referenced
this issue
Mar 10, 2017
added a commit
to arielb1/rust
that referenced
this issue
Mar 10, 2017
added a commit
to alexcrichton/rust
that referenced
this issue
Mar 10, 2017
added a commit
to arielb1/rust
that referenced
this issue
Mar 10, 2017
added a commit
to alexcrichton/rust
that referenced
this issue
Mar 10, 2017
added a commit
to arielb1/rust
that referenced
this issue
Mar 10, 2017
added a commit
to arielb1/rust
that referenced
this issue
Mar 10, 2017
added a commit
to arielb1/rust
that referenced
this issue
Mar 10, 2017
added a commit
to alexcrichton/rust
that referenced
this issue
Mar 10, 2017
added a commit
to alexcrichton/rust
that referenced
this issue
Mar 11, 2017
added a commit
to arielb1/rust
that referenced
this issue
Mar 11, 2017
added a commit
to arielb1/rust
that referenced
this issue
Mar 11, 2017
le-jzr
referenced this issue
in phil-opp/multiboot2-elf64
Apr 12, 2017
Open
repr(C) is necessary #28
|
Actually, it's not even necessary to write So, beyond making references to packed fields unsafe, it seems we also have to forbid types in packed structs that implement Drop. Will this need an RFC (despite being a soundness fix)? |
|
Because references should always be aligned... how are we supposed to get a raw pointer to a given field to check whether it is in fact aligned or even do an unaligned read or write? Doing |
This was referenced Jul 13, 2017
|
@RalfJung huh, interesting. We could amend the existing RFC to cover that case -- i.e., open a PR amending the original text -- but I think it also suffices to open up a Rust issue, tag it as T-lang, and do the FCP there; I feel like it's in the spirit of the existing RFC, so I don't know that a full PR on the RFC repository is needed. (It'd be nice to clarify these procedures, this question comes up somewhat regularly; but the intention is that an RFC, once landed, can be amended in reasonable ways, and any concerns will be addressed and discussed in the tracking issue and -- ultimately -- during stabilization.) |
RalfJung
referenced this issue
Jul 18, 2017
Open
repr(packed) calls drop with unaligned pointer #43306
Mark-Simulacrum
added
C-tracking-issue
and removed
C-enhancement
C-feature-request
labels
Jul 22, 2017
|
Notes from IRC: https://botbot.me/mozilla/rustc/2017-07-31/?msg=89222091&page=2 |
@nikomatsakis Is this still relevant? This sounds like a significant task of its own, should it be a separate issue? |
|
I am working on it right now. |
bstrie
added
the
B-unstable
label
Sep 20, 2017
|
For |
|
That wouldn't work for unsized fields (yes you can have I think we should try to forbid |
huonw commentedJul 16, 2015
•
Edited 1 time
-
pnkfelix
Feb 16, 2017
This is now a tracking issue for RFC 1240, "Taking a reference into a struct marked
repr(packed)should becomeunsafe", but originally it was a bug report that led to the development of that RFC.Original Issue Description
Will print, on playpen:
The assembly for the
okload is:But for the
badone, it isSpecifically, the
movups(unaligned) became amovaps(aligned), but the pointer isn't actually aligned, hence the CPU faults.