Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use more fine grained locks for the dep graph #63756

Open
wants to merge 1 commit into
base: master
from

Conversation

@Zoxc
Copy link
Contributor

commented Aug 20, 2019

Split out from #61845.

r? @michaelwoerister cc @aturon

}

// FIXME: Remove this since it appears to be unused?

This comment has been minimized.

Copy link
@Zoxc

Zoxc Aug 20, 2019

Author Contributor

@michaelwoerister I'm guessing you're the person to ask about this.

This comment has been minimized.

Copy link
@michaelwoerister

michaelwoerister Aug 21, 2019

Contributor

At some point we had debug output for incremental compilation that was using this. If it is really unused now, you can remove it.

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Aug 20, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-08-20T19:20:07.0651330Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-20T19:20:07.0864563Z ##[command]git config gc.auto 0
2019-08-20T19:20:07.1299588Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-20T19:20:07.1362515Z ##[command]git config --get-all http.proxy
2019-08-20T19:20:07.1516728Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63756/merge:refs/remotes/pull/63756/merge
---
2019-08-20T19:20:41.2725184Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-20T19:20:41.2725230Z 
2019-08-20T19:20:41.2725436Z   git checkout -b <new-branch-name>
2019-08-20T19:20:41.2725475Z 
2019-08-20T19:20:41.2725522Z HEAD is now at 5ca57c7a4 Merge 80d345747ebb3d890aa08021b969cd24fd7cc0fa into 5a56e05abd34e1936df74625c1f40cb6fee0cd4a
2019-08-20T19:20:41.2892821Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-20T19:20:41.2895528Z ==============================================================================
2019-08-20T19:20:41.2895586Z Task         : Bash
2019-08-20T19:20:41.2895632Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-20T19:30:46.9097528Z    Compiling arena v0.0.0 (/checkout/src/libarena)
2019-08-20T19:30:50.0986540Z    Compiling syntax_pos v0.0.0 (/checkout/src/libsyntax_pos)
2019-08-20T19:30:57.1455584Z    Compiling rustc_errors v0.0.0 (/checkout/src/librustc_errors)
2019-08-20T19:31:12.5147916Z    Compiling fmt_macros v0.0.0 (/checkout/src/libfmt_macros)
2019-08-20T19:31:35.6448390Z error[E0599]: no method named `borrow` found for type `dep_graph::graph::CurrentDepGraph` in the current scope
2019-08-20T19:31:35.6448944Z    --> src/librustc/dep_graph/graph.rs:612:41
2019-08-20T19:31:35.6449175Z     |
2019-08-20T19:31:35.6449528Z 612 |             debug_assert!(!data.current.borrow().node_to_node_index.contains_key(dep_node));
2019-08-20T19:31:35.6450023Z ...
2019-08-20T19:31:35.6450023Z ...
2019-08-20T19:31:35.6450324Z 942 | pub(super) struct CurrentDepGraph {
2019-08-20T19:31:35.6450655Z     | --------------------------------- method `borrow` not found for this
2019-08-20T19:31:35.6461201Z     |
2019-08-20T19:31:35.6461539Z     = help: items from traits can only be used if the trait is in scope
2019-08-20T19:31:35.6461860Z     = note: the following trait is implemented but not in scope, perhaps add a `use` for it:
2019-08-20T19:31:35.6462102Z             `use std::borrow::Borrow;`
2019-08-20T19:31:50.7942952Z error: aborting due to previous error
2019-08-20T19:31:50.7946791Z 
2019-08-20T19:31:50.7956209Z For more information about this error, try `rustc --explain E0599`.
2019-08-20T19:31:50.9721881Z error: Could not compile `rustc`.
---
2019-08-20T19:32:39.4985343Z == clock drift check ==
2019-08-20T19:32:39.5007344Z   local time: Tue Aug 20 19:32:39 UTC 2019
2019-08-20T19:32:39.6574654Z   network time: Tue, 20 Aug 2019 19:32:39 GMT
2019-08-20T19:32:39.6575475Z == end clock drift check ==
2019-08-20T19:32:40.7149248Z ##[error]Bash exited with code '1'.
2019-08-20T19:32:40.7189082Z ##[section]Starting: Checkout
2019-08-20T19:32:40.7191497Z ==============================================================================
2019-08-20T19:32:40.7191574Z Task         : Get sources
2019-08-20T19:32:40.7191645Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@Zoxc Zoxc force-pushed the Zoxc:sharded-dep-graph-1 branch from 80d3457 to bc5ac9c Aug 21, 2019

@michaelwoerister
Copy link
Contributor

left a comment

Thanks for the PR, @Zoxc!

The changes look correct to me. Do we have evidence that they improve performance?

src/librustc/dep_graph/graph.rs Outdated Show resolved Hide resolved
src/librustc/dep_graph/graph.rs Show resolved Hide resolved

@Zoxc Zoxc force-pushed the Zoxc:sharded-dep-graph-1 branch from bc5ac9c to 9972c69 Aug 25, 2019

@Zoxc

This comment has been minimized.

Copy link
Contributor Author

commented Aug 25, 2019

The changes look correct to me. Do we have evidence that they improve performance?

This PR probably doesn't help. It's just a prerequisite for other stuff which does improve performance.

@JohnCSimon

This comment has been minimized.

Copy link

commented Sep 7, 2019

Ping from triage:
@Zoxc @michaelwoerister - this has sat idle for over nearly two weeks. What else is there to be done for this PR?
Thanks!

@michaelwoerister

This comment has been minimized.

Copy link
Contributor

commented Sep 9, 2019

I've taken a look at the code and it looks correct to me, so now it's more of a design question whether we want to do this or not. Since you seem to be doing an audit on all the locking going, @aturon, I'll let you give the go-ahead.

r? @aturon

@JohnCSimon

This comment has been minimized.

Copy link

commented Sep 14, 2019

Ping from triage
@aturon Can you please review this PR? Thanks.

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.