Const prop: erase all block-only locals at the end of every block #73757

oli-obk · 2020-06-26T09:03:24Z

I messed up this erasure in #73656 (comment). I think it is too fragile to have the previous scheme. Let's benchmark the new scheme and see what happens.

r? @wesleywiser

cc @felix91gr

oli-obk · 2020-06-26T11:34:07Z

@bors try @rust-timer queue

rust-timer · 2020-06-26T11:34:08Z

Awaiting bors try build completion

bors · 2020-06-26T11:34:18Z

⌛ Trying commit 783021cc5ce3268672dbcd67afd91ab61b044cd9 with merge 2d4319b8222eaaca3cec82a0c9c18f3c86284b1c...

wesleywiser · 2020-06-26T13:14:12Z

Can we add a regression test for this?

oli-obk · 2020-06-26T13:20:38Z

The other PR has a bug reproducing the issue. Do you want me to move it here? It doesn't do anything here (and it doesn't do anything anymore on the other PR). It's just a piece of code that const prop has no business of ever "improving".

wesleywiser · 2020-06-26T13:23:25Z

Oh, I see. It only triggers with the deaggregator move up. That's fine then.

wesleywiser · 2020-06-26T13:26:04Z

r=me if the perf looks ok

bors · 2020-06-26T14:31:17Z

☀️ Try build successful - checks-azure
Build commit: 2d4319b8222eaaca3cec82a0c9c18f3c86284b1c (2d4319b8222eaaca3cec82a0c9c18f3c86284b1c)

rust-timer · 2020-06-26T14:31:18Z

Queued 2d4319b8222eaaca3cec82a0c9c18f3c86284b1c with parent 14e65d5, future comparison URL.

felix91gr · 2020-06-26T14:55:22Z

Sees iter_enumerate

@oli-obk you're a genius.

Let's see how the perf turns out, this should definitely be better :3

rust-timer · 2020-06-26T22:31:58Z

Finished benchmarking try commit (2d4319b8222eaaca3cec82a0c9c18f3c86284b1c): comparison url.

felix91gr · 2020-06-27T00:11:35Z

Interesting change. Are we sure we're comparing the right commits? @oli-obk

felix91gr · 2020-06-27T00:14:09Z

Fwiw, here it is v/s its direct parent: https://perf.rust-lang.org/compare.html?start=e093b6525079cb71d4158f97480ac6f6ce311eac&end=2d4319b8222eaaca3cec82a0c9c18f3c86284b1c&stat=instructions%3Au

What a weird change. I don't get it. Is enumerate that much costlier?

src/librustc_mir/transform/const_prop.rs

wesleywiser · 2020-06-27T01:55:42Z

@bors try @rust-timer queue

rust-timer · 2020-06-27T01:55:43Z

Awaiting bors try build completion

bors · 2020-06-27T01:55:55Z

⌛ Trying commit 94b17b74fe809430c5d34f0859a16de474272d8d with merge cd6bfbcb5627d1fba2f5b200a9cf88e52c0aca41...

felix91gr · 2020-06-27T02:50:15Z

src/librustc_mir/transform/const_prop.rs

@@ -334,6 +334,13 @@ impl<'mir, 'tcx> ConstPropagator<'mir, 'tcx> {
        )
        .expect("failed to push initial stack frame");

+        let mut only_propagate_inside_block_locals = BitSet::new_empty(can_const_prop.len());
+        for (l, mode) in can_const_prop.iter_enumerated() {


Ohh! I don't think I realized that doing this construction here was legal. This is nice :)

So, just to check: this code is executed once per block, right? @wesleywiser

Otherwise, I don't think this change is 100% sound.
Around line 1082, we might be removing consts from the const pool that are not from the current block, yet are restricted to propagate only into their own block. If that makes sense?

Well, it passed the tests, so either it's sound, or it threaded the needle through our tests.
Still I think that the failure mode of this expansion to const-prop (if it is buggy) is to propagate less, and not propagate more constants nor the wrong constants. So even if it's buggy, it shouldn't be actually unsound.
To catch those cases we'll need more tests. I hope I can work on that after I finish getting through bureaucracy with uni >.<

bors · 2020-06-27T05:01:19Z

☀️ Try build successful - checks-azure
Build commit: cd6bfbcb5627d1fba2f5b200a9cf88e52c0aca41 (cd6bfbcb5627d1fba2f5b200a9cf88e52c0aca41)

rust-timer · 2020-06-27T05:01:20Z

Queued cd6bfbcb5627d1fba2f5b200a9cf88e52c0aca41 with parent 7750c3d, future comparison URL.

rust-timer · 2020-06-27T08:08:56Z

Finished benchmarking try commit (cd6bfbcb5627d1fba2f5b200a9cf88e52c0aca41): comparison url.

oli-obk · 2020-06-27T08:19:33Z

Better, but still a regression if there are many blocks

felix91gr · 2020-06-27T09:56:08Z

That sounds like it would be quite powerful :)

oli-obk · 2020-06-27T10:18:00Z

I'm working on this

wesleywiser · 2020-06-27T12:01:31Z

Looks like we're ready for another round of benchmarks

@bors try @rust-timer queue

rust-timer · 2020-06-27T12:01:33Z

Awaiting bors try build completion

bors · 2020-06-27T12:01:42Z

⌛ Trying commit 70fdfe2244da7183ff81b0d25c413bcbce92e672 with merge c89495d665db21d0da36eaaec48938e506760d65...

bors · 2020-06-27T15:14:27Z

☀️ Try build successful - checks-azure
Build commit: c89495d665db21d0da36eaaec48938e506760d65 (c89495d665db21d0da36eaaec48938e506760d65)

rust-timer · 2020-06-27T15:14:28Z

Queued c89495d665db21d0da36eaaec48938e506760d65 with parent 394e1b4, future comparison URL.

rust-timer · 2020-06-27T17:56:45Z

Finished benchmarking try commit (c89495d665db21d0da36eaaec48938e506760d65): comparison url.

src/librustc_mir/transform/const_prop.rs

wesleywiser

r=me on the approach since the perf looks good. Feel free to squash everything together if you want or we can also just merge as-is.

oli-obk · 2020-06-28T09:22:44Z

src/librustc_mir/interpret/machine.rs

+    /// Since writing a ZST is not actually accessing memory or locals, this is never invoked
+    /// for ZST reads.
+    #[inline]
+    fn access_local_mut<'a>(


cc @RalfJung some miri engine and machine extensions happen in this PR. They don't change anything, they just allow the const propagator to hook into accessing a local mutably and run some extra code.

Thanks for the heads-up!

oli-obk · 2020-06-28T09:23:22Z

@bors r=wesleywiser

bors · 2020-06-28T09:23:23Z

📌 Commit b7801df1c6814ce748b3a179dc06f69756cabba8 has been approved by wesleywiser

RalfJung · 2020-06-28T09:52:16Z

src/librustc_mir/interpret/machine.rs

@@ -192,6 +192,8 @@ pub trait Machine<'mir, 'tcx>: Sized {
    ) -> InterpResult<'tcx>;

    /// Called to read the specified `local` from the `frame`.
+    /// Since reading a ZST is not actually accessing memory or locals, this is never invoked
+    /// for ZST reads.


To be clear, this was already the case before but now is documented better?

RalfJung · 2020-06-28T09:53:42Z

src/librustc_mir/interpret/place.rs

@@ -740,7 +740,7 @@ where
        // but not factored as a separate function.
        let mplace = match dest.place {
            Place::Local { frame, local } => {
-                match self.stack_mut()[frame].locals[local].access_mut()? {
+                match M::access_local_mut(self, frame, local)? {


There's a risk here of accidentally using the frame method instead of the hook. Is there anything we can do about that?

I think there should at least be a comment at the frame method saying to not call it (except when implementing the hook).

I'll investigate whether we can have something static and otherwise document it properly

I documented it. I don't think it's realistic to statically prevent it being called except by introducing some appropriately named token types.

…ave not been touched.

oli-obk · 2020-06-28T10:05:08Z

@bors r=wesleywiser

bors · 2020-06-28T10:05:09Z

📌 Commit b9f4e0d has been approved by wesleywiser

RalfJung · 2020-06-28T10:19:03Z

src/librustc_mir/interpret/eval_context.rs

@@ -131,6 +131,10 @@ pub enum LocalValue<Tag = ()> {
 }

 impl<'tcx, Tag: Copy + 'static> LocalState<'tcx, Tag> {
+    /// Read the local's value or error if the local is not yet live or not live anymore.
+    ///
+    /// Note: This may only be invoked from the `Machine::access_local` hook and not from


Suggested change

/// Note: This may only be invoked from the `Machine::access_local` hook and not from

/// Note: This must only be invoked from the `Machine::access_local` hook and not from

I had must before 😆 I think may is correct here.

RalfJung · 2020-06-28T10:19:17Z

src/librustc_mir/interpret/eval_context.rs

@@ -143,6 +147,9 @@ impl<'tcx, Tag: Copy + 'static> LocalState<'tcx, Tag> {

    /// Overwrite the local.  If the local can be overwritten in place, return a reference
    /// to do so; otherwise return the `MemPlace` to consult instead.
+    ///
+    /// Note: This may only be invoked from the `Machine::access_local_mut` hook and not from


Suggested change

/// Note: This may only be invoked from the `Machine::access_local_mut` hook and not from

/// Note: This must only be invoked from the `Machine::access_local_mut` hook and not from

@ghost

…arth Rollup of 9 pull requests Successful merges: - rust-lang#73577 (Add partition_point) - rust-lang#73757 (Const prop: erase all block-only locals at the end of every block) - rust-lang#73774 (Make liveness more precise for assignments to fields) - rust-lang#73795 (Add some `const_compare_raw_pointers`-related regression tests) - rust-lang#73800 (Forward Hash::write_iN to Hash::write_uN) - rust-lang#73813 (Rename two `Resolver` traits) - rust-lang#73817 (Rename clashing_extern_decl to clashing_extern_declarations.) - rust-lang#73826 (Fix docstring typo) - rust-lang#73833 (Remove GlobalCtxt::enter_local) Failed merges: r? @ghost

rust-highfive assigned wesleywiser Jun 26, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 26, 2020

oli-obk force-pushed the const_prop_hardening branch from 2e4670a to 783021c Compare June 26, 2020 09:04

oli-obk changed the title ~~Const prop hardening~~ Const prop: erase all block-only locals at the end of every block Jun 26, 2020

oli-obk mentioned this pull request Jun 26, 2020

move Deaggregate pass to post_borrowck_cleanup #73656

Merged

wesleywiser approved these changes Jun 26, 2020

View reviewed changes

felix91gr reviewed Jun 27, 2020

View reviewed changes

src/librustc_mir/transform/const_prop.rs Outdated Show resolved Hide resolved

felix91gr reviewed Jun 27, 2020

View reviewed changes

wesleywiser reviewed Jun 27, 2020

View reviewed changes

src/librustc_mir/transform/const_prop.rs Show resolved Hide resolved

wesleywiser approved these changes Jun 27, 2020

View reviewed changes

oli-obk force-pushed the const_prop_hardening branch from 70fdfe2 to b7801df Compare June 28, 2020 09:20

oli-obk commented Jun 28, 2020

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 28, 2020

RalfJung reviewed Jun 28, 2020

View reviewed changes

Erase all block-only locals at the end of every block, even if they h…

b9f4e0d

…ave not been touched.

oli-obk force-pushed the const_prop_hardening branch from b7801df to b9f4e0d Compare June 28, 2020 10:04

RalfJung reviewed Jun 28, 2020

View reviewed changes

Manishearth mentioned this pull request Jun 28, 2020

Rollup of 9 pull requests #73838

Merged

bors merged commit ccc1bf7 into rust-lang:master Jun 28, 2020

oli-obk mentioned this pull request Jul 6, 2020

[mir-opt] Allow debuginfo to be generated for a constant or a Place #73210

Merged

oli-obk deleted the const_prop_hardening branch March 16, 2021 12:14

	/// Note: This may only be invoked from the `Machine::access_local` hook and not from
	/// Note: This must only be invoked from the `Machine::access_local` hook and not from

Const prop: erase all block-only locals at the end of every block #73757

Const prop: erase all block-only locals at the end of every block #73757

Conversation

oli-obk commented Jun 26, 2020 • edited

oli-obk commented Jun 26, 2020

rust-timer commented Jun 26, 2020

bors commented Jun 26, 2020

wesleywiser commented Jun 26, 2020

oli-obk commented Jun 26, 2020

wesleywiser commented Jun 26, 2020

wesleywiser commented Jun 26, 2020

bors commented Jun 26, 2020

rust-timer commented Jun 26, 2020

felix91gr commented Jun 26, 2020

rust-timer commented Jun 26, 2020

felix91gr commented Jun 27, 2020

felix91gr commented Jun 27, 2020

wesleywiser commented Jun 27, 2020

rust-timer commented Jun 27, 2020

bors commented Jun 27, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bors commented Jun 27, 2020

rust-timer commented Jun 27, 2020

rust-timer commented Jun 27, 2020

oli-obk commented Jun 27, 2020

felix91gr commented Jun 27, 2020

oli-obk commented Jun 27, 2020

wesleywiser commented Jun 27, 2020

rust-timer commented Jun 27, 2020

bors commented Jun 27, 2020

bors commented Jun 27, 2020

rust-timer commented Jun 27, 2020

rust-timer commented Jun 27, 2020

wesleywiser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oli-obk commented Jun 28, 2020

bors commented Jun 28, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oli-obk commented Jun 28, 2020

bors commented Jun 28, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oli-obk commented Jun 26, 2020 •

edited