Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HirIdification: rework Map #59042

Merged
merged 3 commits into from
Apr 25, 2019
Merged

Conversation

ljedrz
Copy link
Contributor

@ljedrz ljedrz commented Mar 9, 2019

The next iteration of HirIdification (#57578).

  • remove NodeId from Entry
  • change Map::map to an FxHashMap<HirId, Entry>
  • base the NodeId Map methods on HirId ones (reverses the current state)
  • HirIdify librustdoc a little bit (some NodeId Map methods were converted to work on HirIds)

The second change might have performance implications, so I'd do a perf run to be sure it's fine; it simplifies the codebase and shouldn't have an impact as long as the Map searches are cached (which is now possible thanks to using HirIds).

r? @Zoxc

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 9, 2019
@ljedrz
Copy link
Contributor Author

ljedrz commented Mar 9, 2019

@bors try (for perf)

@bors
Copy link
Contributor

bors commented Mar 9, 2019

⌛ Trying commit 12c55fd with merge 920010c...

bors added a commit that referenced this pull request Mar 9, 2019
HirIdification: rework Map

The next iteration of HirIdification (#57578).

- remove `NodeId` from `Entry`
- change `Map::map` to an `FxHashMap<HirId, Entry>`
- base the `NodeId` `Map` methods on `HirId` ones (reverses the current state)
- HirIdify `librustdoc` a little bit (some `NodeId` `Map` methods were converted to work on `HirId`s)

The second change might have performance implications, so I'd do a perf run to be sure it's fine; it simplifies the codebase and shouldn't have an impact as long as the `Map` searches are cached (which is now possible thanks to using `HirId`s).

r? @Zoxc
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:03f1a5dc:start=1552121008554060787,finish=1552121082514661400,duration=73960600613
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
Setting environment variables from .travis.yml
---
[00:59:57]    Compiling rustdoc v0.0.0 (/checkout/src/librustdoc)
[01:00:03] error[E0308]: mismatched types
[01:00:03]   --> src/librustdoc/passes/collect_intra_doc_links.rs:82:49
[01:00:03]    |
[01:00:03] 82 |                                     .with_scope(id,
[01:00:03]    |                                                 ^^ expected struct `syntax::ast::NodeId`, found struct `rustc::hir::HirId`
[01:00:03]    = note: expected type `syntax::ast::NodeId`
[01:00:03]               found type `rustc::hir::HirId`
[01:00:03] 
[01:00:03] error[E0308]: mismatched types
[01:00:03] error[E0308]: mismatched types
[01:00:03]    --> src/librustdoc/passes/collect_intra_doc_links.rs:146:45
[01:00:03]     |
[01:00:03] 146 |                                 .with_scope(id,
[01:00:03]     |                                             ^^ expected struct `syntax::ast::NodeId`, found struct `rustc::hir::HirId`
[01:00:03]     = note: expected type `syntax::ast::NodeId`
[01:00:03]                found type `rustc::hir::HirId`
[01:00:03] 
[01:00:04] error[E0308]: mismatched types
[01:00:04] error[E0308]: mismatched types
[01:00:04]    --> src/librustdoc/visit_ast.rs:275:39
[01:00:04]     |
[01:00:04] 275 |                 if cx.tcx.hir().attrs(node).lists("doc").has_word("hidden") {
[01:00:04]     |                                       ^^^^ expected struct `syntax::ast::NodeId`, found struct `rustc::hir::HirId`
[01:00:04]     = note: expected type `syntax::ast::NodeId`
[01:00:04]                found type `rustc::hir::HirId`
[01:00:04] 
[01:00:04] error: aborting due to 3 previous errors
---
travis_time:end:097e7ef8:start=1552124696918302332,finish=1552124696923750692,duration=5448360
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:2aca16b6
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:1568c58c
travis_time:start:1568c58c
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Mar 9, 2019

💔 Test failed - checks-travis

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 9, 2019
@rust-highfive
Copy link
Collaborator

The job dist-x86_64-linux of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_fold:end:services

travis_fold:start:git.checkout
travis_time:start:15d26819
$ git clone --depth=2 --branch=try https://github.com/rust-lang/rust.git rust-lang/rust
---
[01:16:00]    Compiling rustdoc v0.0.0 (/checkout/src/librustdoc)
[01:16:05] error[E0308]: mismatched types
[01:16:05]   --> src/librustdoc/passes/collect_intra_doc_links.rs:82:49
[01:16:05]    |
[01:16:05] 82 |                                     .with_scope(id,
[01:16:05]    |                                                 ^^ expected struct `syntax::ast::NodeId`, found struct `rustc::hir::HirId`
[01:16:05]    = note: expected type `syntax::ast::NodeId`
[01:16:05]               found type `rustc::hir::HirId`
[01:16:05] 
[01:16:05] error[E0308]: mismatched types
[01:16:05] error[E0308]: mismatched types
[01:16:05]    --> src/librustdoc/passes/collect_intra_doc_links.rs:146:45
[01:16:05]     |
[01:16:05] 146 |                                 .with_scope(id,
[01:16:05]     |                                             ^^ expected struct `syntax::ast::NodeId`, found struct `rustc::hir::HirId`
[01:16:05]     = note: expected type `syntax::ast::NodeId`
[01:16:05]                found type `rustc::hir::HirId`
[01:16:05] 
[01:16:06] error[E0308]: mismatched types
[01:16:06] error[E0308]: mismatched types
[01:16:06]    --> src/librustdoc/visit_ast.rs:275:39
[01:16:06]     |
[01:16:06] 275 |                 if cx.tcx.hir().attrs(node).lists("doc").has_word("hidden") {
[01:16:06]     |                                       ^^^^ expected struct `syntax::ast::NodeId`, found struct `rustc::hir::HirId`
[01:16:06]     = note: expected type `syntax::ast::NodeId`
[01:16:06]                found type `rustc::hir::HirId`
[01:16:06] 
[01:16:06] error: aborting due to 3 previous errors
---
travis_time:end:29603c5a:start=1552125624405458878,finish=1552125624414692049,duration=9233171
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:060a2f09
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:0e52fe18
travis_time:start:0e52fe18
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:057ad52b
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@ljedrz ljedrz force-pushed the HirIdification_rework_map branch from 12c55fd to 16b3081 Compare March 9, 2019 10:23
@ljedrz
Copy link
Contributor Author

ljedrz commented Mar 9, 2019

Oof

@bors try (for perf, for real this time)

@bors
Copy link
Contributor

bors commented Mar 9, 2019

⌛ Trying commit 16b3081 with merge 3c13d8d...

bors added a commit that referenced this pull request Mar 9, 2019
HirIdification: rework Map

The next iteration of HirIdification (#57578).

- remove `NodeId` from `Entry`
- change `Map::map` to an `FxHashMap<HirId, Entry>`
- base the `NodeId` `Map` methods on `HirId` ones (reverses the current state)
- HirIdify `librustdoc` a little bit (some `NodeId` `Map` methods were converted to work on `HirId`s)

The second change might have performance implications, so I'd do a perf run to be sure it's fine; it simplifies the codebase and shouldn't have an impact as long as the `Map` searches are cached (which is now possible thanks to using `HirId`s).

r? @Zoxc
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:111a9abc:start=1552127067054046443,finish=1552127142662952951,duration=75608906508
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
Setting environment variables from .travis.yml
---
travis_time:start:test_debuginfo
Check compiletest suite=debuginfo mode=debuginfo-both (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:27:42] 
[01:27:42] running 119 tests
[01:28:10] .iiiii...i.....i..i...i..i.i..i.ii...i.....i..i....i..........iiii..........i...ii...i.......ii.i.i. 100/119
[01:28:14] i......iii.i.....ii
[01:28:14] 
[01:28:14]  finished in 32.648
[01:28:14] travis_fold:end:test_debuginfo

---
[01:52:03]     Finished release [optimized] target(s) in 47.88s
[01:52:03]      Running build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc_driver-72f48af8359eb542
[01:52:03] 
[01:52:03] running 9 tests
[01:52:03] ...F.....
[01:52:03] 
[01:52:03] ---- test::contravariant_region_ptr_err stdout ----
[01:52:03] ---- test::contravariant_region_ptr_err stdout ----
[01:52:03] thread 'test::contravariant_region_ptr_err' panicked at 'index out of bounds: the len is 1 but the index is 4294967040', /checkout/src/libcore/slice/mod.rs:2539:10
[01:52:03] 
[01:52:03] 
[01:52:03] failures:
[01:52:03]     test::contravariant_region_ptr_err
[01:52:03]     test::contravariant_region_ptr_err
[01:52:03] 
[01:52:03] test result: FAILED. 8 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out
[01:52:03] 
[01:52:03] error: test failed, to rerun pass '--lib'
[01:52:03] 
[01:52:03] 
[01:52:03] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--locked" "--color" "always" "--features" "" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "-p" "rustc_driver" "--" "--quiet"
[01:52:03] 
[01:52:03] 
[01:52:03] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:52:03] Build completed unsuccessfully in 0:37:41
[01:52:03] Build completed unsuccessfully in 0:37:41
[01:52:03] make: *** [check] Error 1
[01:52:03] Makefile:48: recipe for target 'check' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0bfdefd4
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Sat Mar  9 12:17:55 UTC 2019
---
travis_time:end:06567592:start=1552133877784669710,finish=1552133877790069673,duration=5399963
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:020e05ea
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Mar 9, 2019

☀️ Try build successful - checks-travis
Build commit: 3c13d8d

@ljedrz
Copy link
Contributor Author

ljedrz commented Mar 9, 2019

Can I get a perf run pls?

@Xanewok
Copy link
Member

Xanewok commented Mar 9, 2019

Assuming I remember the invocation...

@rust-timer build 3c13d8d

@Mark-Simulacrum
Copy link
Member

@rust-timer build 3c13d8d

@rust-timer
Copy link
Collaborator

Success: Queued 3c13d8d with parent c9f8304, comparison URL.

@Xanewok
Copy link
Member

Xanewok commented Mar 9, 2019

@Mark-Simulacrum huh so only bare command works in the comment body?

@Mark-Simulacrum
Copy link
Member

No, it should've responded to you somehow. I need to investigate why it didn't.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit 3c13d8d

@ljedrz
Copy link
Contributor Author

ljedrz commented Mar 9, 2019

Hmm, looks like there are some instructions and max-rss losses.

@Zoxc do you think we should also give it a shot together with #57173 or that the map should remain a Vec? I'm not sure how dense HirIds are; if they are not dense enough, perf results would probably be similar. Actually I'm not even sure if it's possible to index a Vec with HirIds; we could always perform a HirId to NodeId conversion for that, but I'm not sure if that's the best course of action - we would want to deprecate those as much as possible after all.

@Zoxc
Copy link
Contributor

Zoxc commented Mar 9, 2019

I would wait until #58623 lands before trying to replace the existing node map. Until then I'd just focus on getting rid of NodeId uses outside the HIR map.

@ljedrz
Copy link
Contributor Author

ljedrz commented Mar 9, 2019

Ok; can you mark this PR as blocked so it doesn't pop up during PR queue cleanups?

@Zoxc Zoxc added S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 9, 2019
@Amanieu
Copy link
Member

Amanieu commented Mar 10, 2019

Is it really necessary to wait for #58623 to land? That PR only touches libstd and doesn't really interact with this PR. The HashMap API is completely unchanged, just the implementation.

@bors
Copy link
Contributor

bors commented Apr 24, 2019

📌 Commit 37954df has been approved by Zoxc

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. labels Apr 24, 2019
@bors
Copy link
Contributor

bors commented Apr 24, 2019

⌛ Testing commit 37954df with merge 49c193172ef3ae229b5c8149a67ff5e31c0d4672...

@ljedrz
Copy link
Contributor Author

ljedrz commented Apr 24, 2019

I mean the instruction results did seem to improve in comparison to the previous perf run; did you expect greens in the incremental department?

@Zoxc
Copy link
Contributor

Zoxc commented Apr 24, 2019

@ljedrz wall-time is the metric which counts, which seems pretty much as bad as before.

@pietroalbini
Copy link
Member

Yielding priority to the stable release.

@bors retry

@Zoxc Zoxc mentioned this pull request Apr 24, 2019
@bors
Copy link
Contributor

bors commented Apr 24, 2019

⌛ Testing commit 37954df with merge 04480fd5dd4945b4a4bb31e46934bff1739df63b...

@pietroalbini
Copy link
Member

Yielding priority to the stable release, again.

@bors retry

@bors
Copy link
Contributor

bors commented Apr 25, 2019

⌛ Testing commit 37954df with merge 3d21124...

bors added a commit that referenced this pull request Apr 25, 2019
HirIdification: rework Map

The next iteration of HirIdification (#57578).

- remove `NodeId` from `Entry`
- change `Map::map` to an `FxHashMap<HirId, Entry>`
- base the `NodeId` `Map` methods on `HirId` ones (reverses the current state)
- HirIdify `librustdoc` a little bit (some `NodeId` `Map` methods were converted to work on `HirId`s)

The second change might have performance implications, so I'd do a perf run to be sure it's fine; it simplifies the codebase and shouldn't have an impact as long as the `Map` searches are cached (which is now possible thanks to using `HirId`s).

r? @Zoxc
@bors
Copy link
Contributor

bors commented Apr 25, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: Zoxc
Pushing 3d21124 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Apr 25, 2019
@bors bors merged commit 37954df into rust-lang:master Apr 25, 2019
@bors bors mentioned this pull request Apr 25, 2019
@rust-highfive
Copy link
Collaborator

📣 Toolstate changed by #59042!

Tested on commit 3d21124.
Direct link to PR: #59042

💔 clippy-driver on windows: test-pass → build-fail (cc @Manishearth @llogiq @mcarton @oli-obk @phansch, @rust-lang/infra).
💔 clippy-driver on linux: test-pass → build-fail (cc @Manishearth @llogiq @mcarton @oli-obk @phansch, @rust-lang/infra).
💔 rls on windows: test-fail → build-fail (cc @Xanewok, @rust-lang/infra).
💔 rls on linux: test-fail → build-fail (cc @Xanewok, @rust-lang/infra).

rust-highfive added a commit to rust-lang-nursery/rust-toolstate that referenced this pull request Apr 25, 2019
Tested on commit rust-lang/rust@3d21124.
Direct link to PR: <rust-lang/rust#59042>

💔 clippy-driver on windows: test-pass → build-fail (cc @Manishearth @llogiq @mcarton @oli-obk @phansch, @rust-lang/infra).
💔 clippy-driver on linux: test-pass → build-fail (cc @Manishearth @llogiq @mcarton @oli-obk @phansch, @rust-lang/infra).
💔 rls on windows: test-fail → build-fail (cc @Xanewok, @rust-lang/infra).
💔 rls on linux: test-fail → build-fail (cc @Xanewok, @rust-lang/infra).
phansch added a commit to phansch/rust-clippy that referenced this pull request Apr 25, 2019
bors added a commit to rust-lang/rust-clippy that referenced this pull request Apr 25, 2019
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Apr 25, 2019
Changes:
````
Rustup for rust-lang#59042
Update pulldown_cmark to 0.5
Only run AppVeyor on r+, try and the master branch
Remove approx_constant known problems
Suppress let_and_return if let has attributes
Add test for or_fun_call macro suggestion
UI test cleanup: Extract needless_range_loop tests
Change "if types change" to "if you later change the type"
````
bors added a commit that referenced this pull request Apr 25, 2019
submodules: update clippy from 9897442 to 8c0e038

Should fix clippy/rls toolstate breakage

Changes:
````
Rustup for #59042
Update pulldown_cmark to 0.5
Only run AppVeyor on r+, try and the master branch
Remove approx_constant known problems
Suppress let_and_return if let has attributes
Add test for or_fun_call macro suggestion
UI test cleanup: Extract needless_range_loop tests
Change "if types change" to "if you later change the type"
````

r? @oli-obk
bors added a commit that referenced this pull request Apr 25, 2019
submodules: update clippy from 9897442 to 8c0e038

Should fix clippy/rls toolstate breakage

Changes:
````
Rustup for #59042
Update pulldown_cmark to 0.5
Only run AppVeyor on r+, try and the master branch
Remove approx_constant known problems
Suppress let_and_return if let has attributes
Add test for or_fun_call macro suggestion
UI test cleanup: Extract needless_range_loop tests
Change "if types change" to "if you later change the type"
````

r? @oli-obk
bors added a commit that referenced this pull request May 8, 2019
@ljedrz ljedrz deleted the HirIdification_rework_map branch June 24, 2019 19:05
flip1995 pushed a commit to flip1995/rust-clippy that referenced this pull request May 5, 2020
Changes:
````
Rustup for rust-lang/rust#59042
Update pulldown_cmark to 0.5
Only run AppVeyor on r+, try and the master branch
Remove approx_constant known problems
Suppress let_and_return if let has attributes
Add test for or_fun_call macro suggestion
UI test cleanup: Extract needless_range_loop tests
Change "if types change" to "if you later change the type"
````
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants