RFC: hint::black_box

[gnzlbg]

Adds black_box to core::hint.

Rendered.

[sfackler]

Probably worth adding why these are useful for benchmarks to the motivation section.

[gnzlbg]

Yeah, that was a bit scarce. I've expanded the motivation.

[rkruppe]

Nit, but this doesn't really match what the implementation of black_box does (and what the linked godbolt shows). It forces the value into memory, and any register traffic is due to that (e.g. in the linked godbolt, the add edi, 2 is not a dead definition, it's immediately stored to [rbp - 4], and that store is dead).

[gnzlbg]

@rkruppe how would you formulate this for the guide-level explanation? Maybe I can just say that 4 must be stored into memory and leave it at that.

[rkruppe]

That seems decent. But it's tricky to say anything about this sort of feature, because it's inherently very tied to optimizer capabilities and the resulting machine code =/

[gnzlbg]

So I've reworded this part a bit, but maybe for the guide level explanation we might want to be even more vague and just leave it at: "the value cannot be discarded" or "prevents the value x from being optimized away".

[rkruppe]

This wording (flushing pending writes) makes me uncomfortable because it's remniscient of memory consistency models, including hardware ones, when this is just a single-threaded and compiler-level restriction. Actually, come to think of it, I'd like to know the difference between this and compiler_fence(SeqCst). I can't think of any off-hand.

[gnzlbg]

when this is just a single-threaded and compiler-level restriction

This is correct.

compiler_fence(SeqCst). I can't think of any off-hand.

I can't either, but for some reason they do generate different code: https://godbolt.org/g/G2UoZC

I'll give this some more thought.

[nagisa]

The difference between asm! with a memory clobber and compiler_fence exists in the fact, that memory clobber requires compiler to actually reload the memory if they want to use it again (as memory is… clobbered – considered changed), whereas compiler_fence only enforces that memory accesses are not reordered and the compiler still may use the usual rules to figure that it needn’t to reload stuff.

[gnzlbg]

@nagisa the only thing that clobber should do is flush pending writes to memory. It doesn't need to require that the compiler reloads memory on reuse. Maybe the volatile asm! with memory clobber is not the best way to implement that.

[rkruppe]

@nagisa Thank you. I was mislead by the fact that fences prohibit some load-store optimization to thinking they'd also impact things like store-load forwarding on the same address with no intervening writes.

@gnzlbg asm! with memory is simply assumed to read from and write to all memory and all its effects follow from that. However, to be precise, this does not mean any reloads are introduced after a clobbering inline asm, it just means that all the loads that is already there (of which are a lot given that every local and many temporaries are stack slots) can't be replaced with values loaded from the same address before the clobber.

If you want something less strong, you need to be precise. "Flushing loads writes" is not really something that makes intuitive sense at the compiler level (and it surely has no effect on runtime, for example caches or store buffers?).

[gnzlbg]

@rkruppe mem::clobber() should be assumed to read from all memory with effects, so that any pending memory stores must have completed before the clobber. All the loads that are already there in registers, temporary stack slots, etc, should not be invalidated by the clobber.

[rkruppe]

Okay that is a coherent concept. I'm not sure off-hand how to best implement that in LLVM. (compiler_fence is probably not enough, since it permits dead store elimination if the memory location is known to not escape.)

[rkruppe]

However, come to think of it, what's the difference between black_box(x) (which is currently stated to just force a write of x to memory) and let tmp = x; clobber(); (which writes x to the stack slot of tmp and then forces that store to be considered live)?

[gnzlbg]

That's a good question and this relates to what I meant with "block scope" . In

{ 
    let tmp = x; 
    clobber();
}

this {} block is the only part of the program that knows the address of tmp so no other code can actually read it without invoking undefined behavior. So in this case, clobber does not make the store of tmp live, because nothing outside of this scope is able to read from it, and this scope doesn't read from it but executes clobber instead (clobbering "memory" does not clobber temporaries AFAICT).

However, if one shares the address of tmp with the rest of the program:

{ 
    let tmp = x; 
    black_box(&tmp);
    clobber();
}

then clobber will force the store of tmp to be considered live.

[leodasvacas]

does the black box need to be inside each iteration?

[gnzlbg]

Not in this case because the vector does not grow. If the vector would be allowed to grow on push, then the answer would be yes.

[rkruppe]

I was already concerned about the interaction with the memory model, and this comment basically confirmed my worst suspicions: There are extremely subtle interactions between the memory model and what these function can do.

I'm starting to believe that we'll be unable to give any guarantes that are of any use to benchmark authors. But if this is the case, these functions become a matter of chasing the optimizer and benchmarking reliably requires checking the asm to see your work wasn't optimize out. That's clearly very unappealing, but has always been true of micro benchmarks, so maybe that's just unavoidable.

This also raises the question of why this needs to be in std, if it's not compiler magic (like compiler_fence is) but a natural consequence of the memory model and inline asm. Aside from the stability of inline asm, which is not a very convincing argument to me.

[gnzlbg]

I'm temporarily closing this till we resolve the semantics of these functions in the memory model repo.

[gnzlbg]

The issue in the memory model repo is: nikomatsakis/rust-memory-model#45

[gnzlbg]

Update. The summary of the discussion on the memory model repo is that clobber should be scrapped, and the RFC updated with the definition of black box agreed on there.

[Manishearth]

Any updates on opening the new RFC?

[gnzlbg]

I've updated the RFC with the discussion of the memory model repo.

It now just proposes pub unsafe fn core::hint::black_box<T>(x: T) -> T, which is specified as an unknown unsafe function that returns x. It is a no-op in the virtual machine, but the compiler has to assume that it can perform any valid operation on x that unsafe Rust code is allowed to perform.

Other changes:

• moved the function from core::mem to core::hint since that appears to be a more suitable place for this functionality which is a no-op.

Open questions:

• do we need a "safe" alternative? What safe and unsafe Rust are allowed to do differs, e.g., if the user passes it a raw pointer unsafe Rust is allowed to dereference it while safe Rust is not.

• In the memory model repo discussion it was a bit unclear (to me) whether this function should take a reference or not. I've keep the interface from the test crate which takes a value, which allows users to pass it a &/&mut/*mut... or whatever they want, but I am not sure this is the right call.

• should we call this black_box or something else ? (e.g. unknown)

cc @RalfJung @ubsan @rkruppe

[Manishearth]

Unsure if we should be opening a new RFC or reopening this one, cc @rust-lang/libs

[rkruppe]

"4 must be stored into memory" does not follow from the definition of black_box given here, it only has to be materialized in some way (e.g., in a register specified by the calling convention). Of course, the current (and likely most practical) implementation using inline assembly does force the value into memory, but that's neither here nor there.

[RalfJung]

Which virtual machine?

I know what you mean but many readers will likely be confused.

[gnzlbg]

Yeah I don't think there is a good way to put it right now beyond maybe saying "abstract" instead of "virtual".

If we had a memory model, that memory model would specify the abstract machine that Rust runs on and on which the memory model is valid, and I could just refer to that abstract machine here.

Maybe I can rephrase that as "is a no-op in the abstract machine of Rust's memory model (whatever that memory model might end up being)" or something like that ?

[RalfJung]

I don't think most people will know what this means.

I would say something like "You can rely on black_box being a NOP just returning x, but the compiler will optimize under the pessimistic assumption that black_box might do anything with the data it got".

[RalfJung]

The emphasis here is on allowed to -- as opposed to "any possible way that unsafe Rust could use x". I think this should be made clearer. Maybe link to the nomicon?

Actually, "any possible way that safe Rust could use x" is almost more precise, but may also give the wrong impression.

[gnzlbg]

The emphasis here is on allowed to -- as opposed to "any possible way that unsafe Rust could use x". I think this should be made clearer. Maybe link to the nomicon?

Thanks, I think this makes more sense.

Actually, "any possible way that safe Rust could use x" is almost more precise, but may also give the wrong impression.

How so? I thought unsafe Rust was allowed to use unsafe for, e.g., dereferencing raw pointers, while if this would be a safe function, it couldn't do so. In any case, whatever this function does, it cannot invoke undefined behavior and the compiler is allowed to assume that.

[RalfJung]

Seems fine to me! I was worried you'd try to talk about which memory black_box could touch, but you actually are using the spec I was hoping for. :)

[comex]

A good point of comparison might be #[inline(always)] and #[inline(never)]. Similar to black_box, those attributes are only meaningful in a specific kind of instantiation of the Rust virtual machine: one that compiles to some sort of assembly or machine code. They're not meaningful in miri; rustc can inline functions at the MIR level but currently treats inline(always) as a hint, which is probably fine as people don't expect miri to have predictable performance characteristics. On the other hand, when compiling to assembly, they are not optional and not just a hint; they have a specific effect, which IMO you should be able to rely on in any fully conformant Rust implementation that targets assembly.

There are other hypothetical targets where it's ambiguous what the attributes should mean. For example, in a Rust implementation targeting the JVM, should #[inline(never)] merely ensure that the Rust compiler doesn't inline the function into others in the emitted JVM bytecode? Or should it also ensure that the JVM itself doesn't inline it when JITting – and since that's basically impossible, should the attribute not be supported? I don't think we need to answer that question now, but I do think it ideally should be answered in the event someone actually creates a JVM implementation.

Similarly, you could have an implementation where the meaning is unambiguous but the implementation strategy makes the effect impossible to achieve. For example, a portable Rust-to-assembly compiler could be implemented by compiling to C and then calling cc to produce assembly. (Not entirely hypothetical, since mrustc compiles to C.) If it stuck to standard C rather than compiler-specific extensions, it would have no way to guarantee the presence or absence of inlining. However, I think such an implementation should be considered not fully conformant. (Not the end of the world in this case, since compiling to standard C would force many other limitations as well.)

edit: In the case of black_box, guaranteeing an effect at the assembly level (which I think it should) is different from guaranteeing timing. After all, even if the backend knows nothing about a value, there's no rule preventing it from performing a transformation like

foo(x * 2);

->

if x == 3 {
    foo(6);
} else {
    foo(x * 2);
}

which could cause observable timing effects depending on the value. And then of course there's the hardware, which does who knows what.

[Centril]

With the recent round of changes in ea2dfeb, satisfying my notes in #2360 (comment) and #2360 (comment) with respect to guarantees, I believe the language team's interest and business with respect to this RFC is concluded.

[gnzlbg]

@Centril I hope the last commit incorporates all of your feedback.

@RalfJung it might be worth going through this again :/

@hdevalence Is the requirement for constant-time machine code binary (yes/no) or does the requirement "a certain degree of constant-timeness" also exist? In practice, black_box might appear to give you a certain degree of constant-timeness, but the only tool that can come close to guarantee constant-time machine code is manually hand-written assembly (black_box, nightly Rust, C, etc. all appear to be too unreliable for this purpose right now).

In the spec, however, black_box is now just an identity function. All optimization-related "effects" are just something that implementations are encouraged but not required to do. These "effects" are not even implementation defined, that is, implementations are not required to state what they actually do here. That is, if you want to write constant time code, the reference of black_box now explicitly states that it does not have your back... I don't know much about constant-time code, but I hope that the information about what black_box guarantees is at least useful for evaluating whether you should use it or not.

[Centril]

@gnzlbg Yes, thank you; I think you've captured the important points :)

[hdevalence]

As @gnzlbg described originally,

We can do that, but then the specification of black_box becomes identical to the specification of the identity function and black_box does not solve the problem it aims to solve anymore.

The proposal seems to now instead be that, "in practice" (i.e., in the Rust implementation which actually exists), this identity function will really act as an unknown function, and users are encouraged to reason about this identity function as if it were an unknown function, i.e., as if it had properties which it doesn't actually have.

Specifying black_box as the identity function, rather than an unknown function, removes the entire purpose of having a Rust-supplied black_box function, which I think is really unfortunate.

Finally, I'm sorry to have referenced this issue from an external repo, which has apparently caused the discussion to be derailed into whether black_box gives or should give timing guarantees (no) based on a shallow misunderstanding of a one-sentence placeholder issue I made to stay organized. I will avoid making the same mistake in the future.

[gnzlbg]

@hdevalence

The proposal seems to now instead be that, "in practice" (i.e., in the Rust implementation which actually exists), this identity function will really act as an unknown function, and users are encouraged to reason about this identity function as if it were an unknown function, i.e., as if it had properties which it doesn't actually have.

The reference-level explanation states the semantics of black_box that one can rely on, and while it mentions extra effects that an implementation is encouraged to provide, these are not a requirement.

That is, the Rust implementation that actually exists does not need to, and cannot provide, the same special effects for all targets (e.g. NVPTX, WASM, ASM.js, etc. already define black_box to the identity because LLVM inline assembly support for these isn't great), or even all compilation options (e.g. for -C opt-level=0 there might not be any optimizations to disable).

What this means for users is that they cannot rely on black_box special properties for portable code. Many applications do not need to rely on these special effects portably (e.g. synthetic benchmarks) and black_box is a good choice for those. If an application needs to rely on these special effects, it might still be able to do so for some targets (e.g. if you never compile it for WASM), compilation options (release builds), compiler versions, etc. but this application will be brittle, non-portable, etc. so "use at your own risk" and YMMV.

[RalfJung]

Aside: Cool :) I recall you having formally verified some bits? Link to a paper to study if so?

Our formal language is unfortunately still further from a "small MIR" than I'd like. It's on my TODO list but unlikely to happen before my thesis.

Yeah, I agree -- things are getting very subtle here. AFAIK it is an open problem how to formally specify a language and a notion of compiler correctness that is suited to the needs to cryptographic code (constant-time execution, need to zero memory even when it is immediately thereafter deallocated, stuff like that).

Yeah; I don't have any expertise here; are you aware of some good reading material (papers?) for this?

I am not aware of any papers, it's an open problem after all. ;) The various special problems crypto code has sure have their individual descriptions somewhere, here's one for zeroing memory.

Finally, I'm sorry to have referenced this issue from an external repo, which has apparently caused the discussion to be derailed into whether black_box gives or should give timing guarantees (no) based on a shallow misunderstanding of a one-sentence placeholder issue I made to stay organized. I will avoid making the same mistake in the future.

It's fine and I don't think it was very derailed, most of the discussion was totally relevant. Links are great :)

[RalfJung]

@RalfJung it might be worth going through this again :/

I did, and it looks good!

[liigo]

Since hint::black_box has nothing to do with boxed::Box and keyword box, please avoid using box here.
Use something else, for example, black_hole.

[GabrielMajeri]

@liigo The term black box is often used when talking about testing or software development. The fact that it seems to link up with Box is just an unfortunate coincidence. AFAIK there is no other synonym as well known as "black box (testing)".

And "black hole" doesn't suggest the value goes through (an unspecified process) unharmed, just that it goes to get destroyed.

[gnzlbg]

Google Benchmark calls this DoNotOptimize so that might be a direction worth exploring. The intent is for black_box to behave like an unknown function in practice, so that might be another direction worth exploring (e.g. hint::unknown).

[Centril]

On the bikeshed side, I think black_box is a good name that we should keep, but if that doesn't work for whatever reason, I propose backup names:

• unprocessed
• unoptimized

[comex]

• I much prefer black_box, as it gives a better intuition for what optimizations the function prevents and doesn't prevent. For example, if it were called unoptimized, you might think that it should somehow de-optimize the entire expression passed as argument, i.e. unoptimized(2 + 2) would compute 2 + 2 at runtime.
• I agree that it should be const, but that that shouldn't block stabilization.

[Centril]

cc @japaric -- How would stabilizing this affect criterion.rs and/or similar frameworks?

[gnzlbg]

How would stabilizing this affect criterion.rs and/or similar frameworks?

It would allow them to be more reliable on stable Rust. Currently, criterion uses test::black_box on nightly, and tries to emulate it on stable, but ideally it shouldn't have to.

[japaric]

@Centril what @gnzlbg said is my understanding as well, but I haven't touch criterion's code in over a year. @bheisler is currently maintaining the crate and its dependencies; maybe they call tell you more.

[bheisler]

@Centril @gnzlbg Yes, that's correct. Criterion.rs emulates black_box using a volatile read (a trick I copied from bencher) unless the user requests the true black_box function using a nightly-only Cargo feature. If we could use black_box on stable, I could remove that workaround.

[gnzlbg]

Any progress on moving this forward? What is blocking it?