Running "x.py check" and "x.py test" in parallel leads to errors

[RalfJung]

Since I started using vscode, it is now often the case that the ./x.py check library/std which vscode runs is still not done when I alt-tab back to my terminal to do (for example) ./x.py test --stage 0 library/core --test-args slice. That regularly leads to errors like this:

error[E0464]: multiple matching crates for `core`
  |
  = note: candidates:
          crate `core`: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-deaf58ff3f911d75.rlib

error: aborting due to previous error

error: could not compile `getrandom`.

In those cases, just running x.py test again solves the problem -- but it would still be better if having vscode run in background would not break what I do in the terminal.

Cc @Mark-Simulacrum

[Mark-Simulacrum]

So there's several solutions to this:

• Don't support this, tell people to run x.py check in a different directory so the build directory goes elsewhere.
• Add a global lock like Cargo's on the build directory
• Specialize x.py check to use different sysroot for stage0 - i.e. don't fix x.py build and test running simultaneously etc

I am interested in hearing which you think would be preferable.

Cc @spastorino

[RalfJung]

If they could run truly in parallel that would be great. However, a separate build dir also means the bootstrap rustc is downloaded again, right? That would not be great.

[Mark-Simulacrum]

Yes, option 1 would duplicate the stage0 rustc downloads. Option 3, though, would not - it's probably the right choice I think since x.py check is somewhat unique in being the most likely to be invoked in parallel.

[jyn514]

@Mark-Simulacrum in an ideal world, would fixing #65427 also fix this?

[Mark-Simulacrum]

Not really. It might even make things worse. rustbuild uses the same stamps for both check and build steps (and sysroots etc) so when both are running simultaneously we will stomp all over the other, which is going to lead to incorrect behavior.

[jyn514]

I was not able to replicate this. On my machine running both in parallel gives

Checking std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Blocking waiting for file lock on build directory

[jyn514]

Ok, I figured out when this happens: you need one terminal to be building the compiler while the other is building libstd. If both are building libstd this error doesn't show up.

[RalfJung]

I think this can also lead to losing the build cache... at least I just had a case where after showing this error, it then (on the next attempt) proceeded to rebuild the crate graph rustc depends on, instead of just the crate that I changed.

[jyn514]

FYI option 3 was seen as too big a hammer in #77779 (comment); the proposed fix was to use a build lock on the sysroot instead, the same way cargo has a build lock for target/deps.

[Mark-Simulacrum]

I think that's not quite accurate, in #77779 the specialization was too specific (i.e., the problem is more widespread, any two x.py runs can touch shared files currently). "Insufficiently big hammer" is probably the right description of option 3.

[worldeva]

@rustbot claim

[RalfJung]

I am afraid this is still not working properly: when I run x.py in the terminal while there is a vscode x.py check running in the background, I get

Waiting for lock on build directory

but once vscode is done, all that happens is that the x.py in the terminal starts using 100% CPU. It does not actually start running though.

[Mark-Simulacrum]

Ah: https://github.com/rust-lang/rust/pull/88310/files#diff-9e25a089f077eae8c8cefe9586cc07498e64cd950be4c954cb90e4245e5f9fc3R42-R46 looks like it missed actually breaking out of that loop -- happy to accept a PR to do so.

[RalfJung]

@worldeva is that something you could take a look at?