Consider allowing reuse of metadata between cargo check and cargo build

[crumblingstatue]

As I understand it, objects generated by cargo build also contain metadata, since it's information required by rustc.

Could it be possible for cargo check to reuse metadata generated by cargo build?

Currently, if you are using cargo check and have a large dependency chain, you effectively have to recompile it twice. Once for metadata-only, once for metadata+codegen. Even if we compile metadata-only, compiling a large dependency chain can still take a significant amount of time. Compiling conrod and its dependencies with cargo check takes more than 3.5 minutes on my system. If cargo check was able to reuse cargo build artifacts, that would shave 3.5 minutes off when I had to recompile the dependency chain (nightly upgrade, etc.).

As an aside, the old cargo-check crate worked this way. You didn't have to recompile dependencies twice in order to use it. But I understand that the new one is more sophisticated, so it doesn't necessarily have to be able to operate this way.

I've also noticed that cargo check also recompiles dependencies between debug and release modes. I don't see why there should be separate metadata for debug and release. It should reuse the same metadata.
EDIT: cfg(debug_assertions) is a thing

I think it's a worthy pursuit to redesign the underlying architecture in order to avoid recompiling as much as possible. After all, that's the whole point of cargo check: To quickly check things without having to wait for things to build.

[alexcrichton]

cc @nrc

We discussed this a bit on the original PR and the implementation is quite non-trivial, but it'd definitely be a welcome improvement!

[crumblingstatue]

Ideally, metadata would be shared between cargo check and cargo build, and even tools like cargo-clippy.

But this might require significant changes in the underlying architecture.

[RalfJung]

We discussed this a bit on the original PR and the implementation is quite non-trivial, but it'd definitely be a welcome improvement!

Do you have a pointer to that discussion? That may be naive, but I kind of thought it couldn't be too hard to consider cargo an rlib as satisfying a dependency on an rmeta. I considered trying to implement this. However, I know nothing about the internal architecture of Cargo.

EDIT: Ah, #3341 has some details. Sounds harder than I had expected indeed.

[alexcrichton]

Weird yeah I can't find what I thought was original-original discussion...

But yeah it ends up being tricky due to how cargo is architected right now :(

[mbrubeck]

#4831 would potentially solve this by causing cargo build to emit metadata files.

[StyMaar]

I was recently surprised by the current behavior when compiling servo : I spent two hours compiling it (yes, in debug mode. I know my computer is getting old) and I was really disappointed when everything needed to be done again when I ran my first cargo check.

[shepmaster]

Ditto for the playground, where this is also an impact on file size. Clippy uses cargo check, so we have to generate an extra 135MB of artifacts and take up an extra 3+ minutes of build time.

[SimonSapin]

Does the recent efforts around pipelined compilation help here? Is or could the metadata file emitted by a pipelined cargo build be at the same location as the one for cargo check?

[ehuss]

Does the recent efforts around pipelined compilation help here?

The .rmeta files created by cargo check are not the same as those created by cargo build (their contents differ). There is a bit of a discussion here about this. I don't know enough about how rustc works to say if it is possible for them to be the same. Perhaps the extra work needed may slow down cargo check which may be undesirable in some cases? Or maybe it's just a bug.

It may be possible for cargo check to reuse the output from cargo build, but not the other way around. However, I think it would add considerable complexity. It would be much simpler if the .rmeta files were the same, and the reuse could go both ways.

[link2xt]

I wrote down some observations for real workspace on the Rust forum:
https://users.rust-lang.org/t/same-dependency-has-four-different-debug-fingerprints/99580

Same non-duplicate dependency generates four metadata files and two .rlib files in debug build.