Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize `shallow_resolve_changed` #67079

Merged
merged 2 commits into from Dec 12, 2019

Conversation

@nnethercote
Copy link
Contributor

nnethercote commented Dec 6, 2019

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 6, 2019

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 6, 2019

Awaiting bors try build completion

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 6, 2019

⌛️ Trying commit 01025ef with merge d4119bb...

bors added a commit that referenced this pull request Dec 6, 2019
…r=<try>

Optimize `shallow_resolve_changed`

r? @nikomatsakis
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 6, 2019

☀️ Try build successful - checks-azure
Build commit: d4119bb (d4119bbbe06d9b4edebef7f71ab6acafff5d0bfa)

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 6, 2019

Queued d4119bb with parent 234c9f2, future comparison URL.

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 6, 2019

Finished benchmarking try commit d4119bb, comparison URL.

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Dec 6, 2019

Seems like it's not really a win, but I'm not opposed to landing it.

@Centril

This comment has been minimized.

Copy link
Member

Centril commented Dec 6, 2019

Seems like a net loss, so why land it?

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 6, 2019

That's weird... it was a clear win of about 3-4% on inflate-check-clean and keccak-check-clean on my machine. I will investigate on Monday.

@nnethercote nnethercote force-pushed the nnethercote:optimize-shallow_resolve_changed branch from 01025ef to 0cb6b45 Dec 9, 2019
@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 9, 2019

I rebased and double-checked the results. Here's what I get locally:

inflate-check
        avg: -4.2%      min: -4.2%      max: -4.2%
keccak-check
        avg: -3.0%      min: -3.0%      max: -3.0%
cranelift-codegen-check
        avg: -0.4%      min: -0.4%      max: -0.4%
clap-rs-check
        avg: -0.3%      min: -0.3%      max: -0.3%
serde-check
        avg: -0.3%      min: -0.3%      max: -0.3%
deeply-nested-check
        avg: -0.2%      min: -0.2%      max: -0.2%
unify-linearly-check
        avg: -0.2%      min: -0.2%      max: -0.2%
wg-grammar-check
        avg: -0.1%      min: -0.1%      max: -0.1%
style-servo-check
        avg: -0.1%      min: -0.1%      max: -0.1%
syn-check
        avg: -0.1%      min: -0.1%      max: -0.1%
deep-vector-check
        avg: -0.1%      min: -0.1%      max: -0.1%
piston-image-check
        avg: -0.1%      min: -0.1%      max: -0.1%
cargo-check
        avg: -0.1%      min: -0.1%      max: -0.1%

Let's try again.

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 9, 2019

Awaiting bors try build completion

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 9, 2019

⌛️ Trying commit 0cb6b45 with merge ec4d9e0...

bors added a commit that referenced this pull request Dec 9, 2019
…r=<try>

Optimize `shallow_resolve_changed`

r? @nikomatsakis
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 9, 2019

☀️ Try build successful - checks-azure
Build commit: ec4d9e0 (ec4d9e01ff378edf7bc46f5d3d0158988ee8fd82)

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 9, 2019

Queued ec4d9e0 with parent 59947fc, future comparison URL.

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 9, 2019

Finished benchmarking try commit ec4d9e0, comparison URL.

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 10, 2019

Finished benchmarking try commit ec4d9e0, comparison URL.

@Mark-Simulacrum: no data is being shown in the results. (Likewise in #66405.) Any idea what's wrong?

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 10, 2019

The new results are finished and they are basically a wash: some slight improvements, some slight regressions. This makes no sense to me.

  • It doesn't match my local results at all.
  • My local results are consistent with the many ObligationForest optimizations I've done before: big wins for inflate and keccak, and smaller wins for cranelift-codegen, clap-rs, and serde.
  • shallow_resolve_unchanged is known to be very hot, and these commits clearly improve it; there are no trade-offs involved, the new code is simply better.

I don't know what to do with this.

nnethercote added 2 commits Dec 6, 2019
It can be made even more specialized.
From a `Vec<Ty>` to a `Vec<InferTy>`, because that's a more restrictive
type. This is a perf win because the ultra-hot function
`shallow_resolve_changed` has less pattern-matching to do.
@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 11, 2019

I did another local perf run, a full one this time. Results are the same as what I got before, and don't match the CI results:

inflate-check
        avg: -3.0%      min: -4.2%      max: 0.0%
keccak-check
        avg: -1.9%      min: -3.0%      max: -0.0%
keccak-debug
        avg: -1.7%      min: -2.6%      max: 0.0%
inflate-debug
        avg: -1.8%      min: -2.6%      max: -0.0%
keccak-opt
        avg: -1.5%      min: -2.3%      max: 0.0%
inflate-opt
        avg: -0.5%?     min: -0.7%?     max: 0.0%?
coercions-debug
        avg: -0.1%?     min: -0.5%?     max: 0.1%?
cranelift-codegen-check
        avg: -0.2%      min: -0.4%      max: 0.0%
clap-rs-check
        avg: -0.1%      min: -0.3%      max: -0.0%
serde-check
        avg: -0.2%      min: -0.3%      max: 0.0%
serde-debug
        avg: -0.2%      min: -0.3%      max: 0.0%
serde-opt
        avg: -0.2%      min: -0.3%      max: 0.0%

Argh.

@nnethercote nnethercote force-pushed the nnethercote:optimize-shallow_resolve_changed branch from 0cb6b45 to 21f35bc Dec 11, 2019
@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 11, 2019

Let's try one more time.

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 11, 2019

Awaiting bors try build completion

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 11, 2019

⌛️ Trying commit 21f35bc with merge 9f4ea4a...

bors added a commit that referenced this pull request Dec 11, 2019
…r=<try>

Optimize `shallow_resolve_changed`

r? @nikomatsakis
@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

Mark-Simulacrum commented Dec 11, 2019

@nnethercote Maybe it's worth trying to benchmark locally with the CI build (i.e., using rustup-toolchain-install-master)? It seems not entirely unlikely that the difference comes down to incremental state or something like that.

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 11, 2019

☀️ Try build successful - checks-azure
Build commit: 9f4ea4a (9f4ea4a85c8473a279666626b279bf8a4ab0d15d)

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 11, 2019

Queued 9f4ea4a with parent 033662d, future comparison URL.

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 11, 2019

Finished benchmarking try commit 9f4ea4a, comparison URL.

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Dec 11, 2019

Having trouble with the comparison URL :( Not sure what's going on with those things

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Dec 11, 2019

Maybe it needs to be rebased so that the comparison is based off a more recent commit?

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 11, 2019

The CI run does two sets of benchmarking, one for the commits, and one for the parent (the baseline). rust-timer reports back when the first set is done, but the URL doesn't display results successfully until the parent is also done. It's in the queue right now, should be finished soon.

I also suspect a bug in rustc-perf... I am also getting mismatches between local results and CI results in #66405. I am investigating now.

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 11, 2019

The latest perf run has finished and the results are closer to what I'd expect (wins on inflate and keccak), but still don't fully match my local results. I'm about to investigate with local runs of the builds used for the CI run.

nnethercote added a commit to nnethercote/rust that referenced this pull request Dec 12, 2019
I have a suspicion that there is a bug in rustc-perf or rust-timer
causing the wrong revisions to be measured by CI. See rust-lang#66405 and rust-lang#67079
for more details.

This commit deliberately causes a massive regression to the
`token-stream-stress` benchmark. On my machine, the instruction count
goes from 313M to 6084M, an 1843.4% regression. I want to see if a CI
run replicates that.
bors added a commit that referenced this pull request Dec 12, 2019
[DO NOT LAND] Regress the `token-stream-stress` benchmark.

I have a suspicion that there is a bug in rustc-perf or rust-timer
causing the wrong revisions to be measured by CI. See #66405 and #67079
for more details.

This commit deliberately causes a massive regression to the
`token-stream-stress` benchmark. On my machine, the instruction count
goes from 313M to 6084M, an 1843.4% regression. I want to see if a CI
run replicates that.

cc @Mark-Simulacrum
r? @ghost
@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 12, 2019

#67248 suggests that CI is working as expected. The most recent CI perf results are good, even though they're not as good as what I saw locally. And logically, the code is clearly better. I will land this, though I'm still uneasy about the variability of the CI perf results.

r=nikomatsakis

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 12, 2019

@bors r=nikomatsakis

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 12, 2019

📌 Commit 21f35bc has been approved by nikomatsakis

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Dec 12, 2019

@bors rollup=never

@mati865

This comment has been minimized.

Copy link
Contributor

mati865 commented Dec 12, 2019

@nnethercote are you building locally with jemalloc enabled? Also difference of glibc versions could make big difference here.

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 12, 2019

⌛️ Testing commit 21f35bc with merge 3eebe05...

bors added a commit that referenced this pull request Dec 12, 2019
…r=nikomatsakis

Optimize `shallow_resolve_changed`

r? @nikomatsakis
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 12, 2019

☀️ Test successful - checks-azure
Approved by: nikomatsakis
Pushing 3eebe05 to master...

@bors bors added the merged-by-bors label Dec 12, 2019
@bors bors merged commit 21f35bc into rust-lang:master Dec 12, 2019
5 checks passed
5 checks passed
homu Test successful
Details
pr #20191211.24 succeeded
Details
pr (Linux mingw-check) Linux mingw-check succeeded
Details
pr (Linux x86_64-gnu-llvm-7) Linux x86_64-gnu-llvm-7 succeeded
Details
pr (Linux x86_64-gnu-tools) Linux x86_64-gnu-tools succeeded
Details
@nnethercote nnethercote deleted the nnethercote:optimize-shallow_resolve_changed branch Dec 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.