Confused about IO safety

[TOETOE55]

AFAIK, safe rust just only guarantees something under the control of the Rust compiler (i.e. rust primitive operations) is sound while external operations like FFI or any other OS features are not guaranteed. For example, to read and write proc/self/mem is safe in rust but it's not actually safe though we have obeyed the safety rules of the fs API. (The safety rules are incomplete.)

I'm confused about IO safety after reading the RFC 3128. What does IO safety actually guaranteed? How does IO safety do it? Why is this sound? And does proc/self/mem violate IO safety?

Thanks.

[RalfJung]

Rust library types can define their own invariants. E.g. you can define a type struct Even(i32) and ensure that it can only ever be constructed for even integers. Then soundness for your particular type is about "being even", even though that is not a concept that the Rust language itself knows about.

The same happens with IO safety: the Rust standard library is using Rust features such as private fields and type invariants to guarantee properties about IO performed through these types/traits. As long as types like OwnedFd are used according to their specs and docs, and programs don't do their own IO bypassing those types (which they are not supposed to do), there is a guarantee that if you own an FD nobody else will mess with it.

[TOETOE55]

Let me take a moment to understand this. IO safety is actually more like a kind of invariant at the "library level", guaranteeing that user or third-party libraries cannot break some internal properties of "owned IO" library.

It seems that It's not the "safety" in usual sense(e.g. dereferring a raw pointer).

[Lokathor]

It's safety in the &str sense: The bytes must be valid utf8 bytes and if that's not the case it's the fault of whoever messed up the bytes, not the person who read the bad bytes and then did an OOB read (or whatever else wrong thing happened)

[RalfJung]

@TOETOE55 you are right that there are basically two levels of safety -- the one hard-coded in the language and understood by the compiler (sometimes called "validity"), and the one expressed as a user-level concept with a type's private invariants. This blog post explains that in some more detail. However, they interact: &str uses the user-defined invariant (that it s UTF-8) to justify certain unsafe operations that would otherwise violate the language-primitive notion of (memory) safety.

IO-safety is of the second, user-defined kind.

[TOETOE55]

@RalfJung Thanks! Maybe I got it.

the one hard-coded in the language and understood by the compiler (sometimes called "validity")

But I still want to confirm whether this case includes FFI(or other external symbols) calls, and whether rust-lang (will) only guarantee the soundness of language-primitive (excluding of FFI calls)?

[CAD97]

Rust (the language) doesn't strictly guarantee anything, because it gives you (the programmer) the power to violate those guarantees with unsafe. Safe Rust (the language subset) does strictly guarantee language-level validity, so long as all unsafe is soundly encapsulated. Safe Rust APIs are considered "sound" if they uphold the guarantees of validity even when actively misused.

What the UCG calls "validity" are the rules that your unsafe MUST obey, on immediate pain of UB if you violate them. These are the primitive guarantees such as byte/layout validity, but the borrow/memory model also classes as a validity requirement. There are also some validity requirements which are highly impractical to violate from user code and are more requirements placed on the host implementation (e.g. that control flow is predictably structured), but anything is theoretically possible when programming the host directly with asm!.

What the UCG calls "safety" is any encapsulation guarantees established by the code in order to provide the soundness guarantee. Violating safety does not necessarily immediately result in "language UB" — what your code does is still consistently defined — but does give the violated library "permission" to arbitrarily violate validity either immediately or whenever in the future. This is colloquially called "library UB;" while some given API usage may not cause language-level UB with these exact library versions, a different, compatible library version may result in upgrading library-level UB to language-level UB.

Remember that UB means that the behavior of your code is fully (and retroactively) unpredictable. Possible results can only be very roughly bounded by what the machine is actually capable of doing, and debuggers may confidently provide inaccurate information, but the most insidious result is that everything works the way you would expect it to (for now). In this way, there's not that much of a difference between "language UB" and "library UB" if you don't control the library.

unsafe code walks the thin line of being able to violate safety, and thus the burden of proving soundness being on the developer. With strong guarantees from the library, you can temporarily suspend safety and e.g. put non-UTF-8 bytes into a String's backing buffer, so long as you can guarantee that the bytes are valid UTF-8 again before invoking any code which might potentially rely on that safety requirement. Controlling the library is the strongest guarantee, as you can rely on the exact implementation, rather than needing documented API guarantees to be able to play looser with safety.

FFI is indeed an interesting case. Rust (the language) knows nothing about FFI, and as such, it is fundamentally unsafe to perform. In the most strictest sense, calling FFI incorrectly is always¹ a safety requirement, because there's no UB caused by evaluation of Rust code; any UB is a result of something which was done on the other side of the FFI bridge. Rust (the language) provides absolutely zero guarantees about the behavior of FFI, and maintaining safety/validity is entirely on the programmer.

But there are things exposed by std which are implemented with FFI, and for essentially everyone except T-lang, T-libs, or subteams thereof, std is essentially part of "the Rust language." If a std API is safe to call (and no unsafe has been used to violate any safety preconditions), then std is providing a guarantee that anything done by the API call is sound. The ability to write to /proc/mem/self is one of a very few minor exceptions, and a concession to being practical over being perfect. A "moral high ground" position would say that a system which allows a /proc/mem/self to be writable is a "non-conformant" host for the Rust std; the more diplomatic is to say that some host capabilities go around the Rust safety model.

To reiterate, soundness of FFI is a soundness guarantee provided by the standard library. Validity is required by the language independent of the library. Since you don't control the library, the only effective differences are that:

• Miri can be used to detect validity problems in pure-Rust code plus a decent portion of host-interacting std, but not soundness problems; and
• Specific library APIs might document careful relaxations in safety requirements when combined with unsafe.

Footnotes

1. Minor nitpick: that the function ABIs are compatible is a validity requirement when the call happens. You can argue that this is a validity requirement imposed by the FFI receiver, though, which leaves getting the call correct as technically only a safety requirement. Just one immediately relied upon for validity. ↩

[RalfJung]

Closing this issue as I hope the question has been answered.

[TOETOE55]

I thought, I already understood, so I wrote an article https://rustmagazine.org/issue-4/a-brief-discussion-on-invariants-in-rust/