Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for -Z binary-dep-depinfo #63012

Open
ehuss opened this issue Jul 26, 2019 · 1 comment
Open

Tracking issue for -Z binary-dep-depinfo #63012

ehuss opened this issue Jul 26, 2019 · 1 comment

Comments

@ehuss
Copy link
Contributor

@ehuss ehuss commented Jul 26, 2019

This is a tracking issue for -Z binary-dep-depinfo added in #61727.
The cargo side is implemented in rust-lang/cargo#7137.

I think the only blocker is dealing with canonicalized paths on Windows. The dep-info file includes a mix of dos-style and extended-length (\\?\) paths, and I think we want to use only one style (whatever is compatible with make and other tools). See the PR for details.

  • Codegen backends are not tracked.

cc @Mark-Simulacrum @alexcrichton

Mark-Simulacrum added a commit to Mark-Simulacrum/rust that referenced this issue Aug 14, 2019
…xcrichton

Utilize -Zbinary-dep-depinfo in rustbuild

The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing).

We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.

Both of these are fixable in the longer term but this existing patch gives us the following benefits:
 * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712.
 * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481.
 * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719.
 * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105.

cc rust-lang#63012
Centril added a commit to Centril/rust that referenced this issue Aug 15, 2019
…xcrichton

Utilize -Zbinary-dep-depinfo in rustbuild

The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing).

We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.

Both of these are fixable in the longer term but this existing patch gives us the following benefits:
 * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712.
 * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481.
 * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719.
 * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105.

cc rust-lang#63012
Centril added a commit to Centril/rust that referenced this issue Aug 15, 2019
…xcrichton

Utilize -Zbinary-dep-depinfo in rustbuild

The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing).

We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.

Both of these are fixable in the longer term but this existing patch gives us the following benefits:
 * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712.
 * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481.
 * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719.
 * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105.

cc rust-lang#63012
Centril added a commit to Centril/rust that referenced this issue Aug 15, 2019
…xcrichton

Utilize -Zbinary-dep-depinfo in rustbuild

The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing).

We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.

Both of these are fixable in the longer term but this existing patch gives us the following benefits:
 * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712.
 * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481.
 * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719.
 * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105.

cc rust-lang#63012
Centril added a commit to Centril/rust that referenced this issue Aug 15, 2019
…xcrichton

Utilize -Zbinary-dep-depinfo in rustbuild

The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing).

We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.

Both of these are fixable in the longer term but this existing patch gives us the following benefits:
 * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712.
 * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481.
 * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719.
 * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105.

cc rust-lang#63012
Centril added a commit to Centril/rust that referenced this issue Aug 15, 2019
…xcrichton

Utilize -Zbinary-dep-depinfo in rustbuild

The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing).

We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.

Both of these are fixable in the longer term but this existing patch gives us the following benefits:
 * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712.
 * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481.
 * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719.
 * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105.

cc rust-lang#63012
bors added a commit that referenced this issue Aug 15, 2019
Utilize -Zbinary-dep-depinfo in rustbuild

The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing).

We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.

Both of these are fixable in the longer term but this existing patch gives us the following benefits:
 * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes #54712.
 * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes #54008, fixes #50481.
 * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes #53284, and fixes #52719.
 * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes #49979 (but no reproduction steps in that issue). Fixes #59105.

cc #63012
bors added a commit that referenced this issue Aug 16, 2019
Utilize -Zbinary-dep-depinfo in rustbuild

The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing).

We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.

Both of these are fixable in the longer term but this existing patch gives us the following benefits:
 * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes #54712.
 * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes #54008, fixes #50481.
 * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes #53284, and fixes #52719.
 * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes #49979 (but no reproduction steps in that issue). Fixes #59105.

cc #63012
bors added a commit that referenced this issue Aug 16, 2019
Utilize -Zbinary-dep-depinfo in rustbuild

We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.

Both of these are fixable in the longer term but this existing patch gives us the following benefits:
 * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes #54712.
 * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes #54008, fixes #50481.
 * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes #53284, and fixes #52719.
 * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes #49979 (but no reproduction steps in that issue). Fixes #59105.

cc #63012
@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

@Mark-Simulacrum Mark-Simulacrum commented Jan 19, 2020

In #68298 we fixed binary dep-depinfo to be less eager to emit dependencies on dylib/rlib files when emitting rlibs and rmeta files, as we only need rmeta input in that case.

It was also noted that we currently do not correctly emit plugin dependencies (I'm not entirely sure of this, but seems not implausible).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.