rustix keeps rebuilding in my dependency tree. #575

rukai · 2023-03-28T23:22:35Z

My application has rustix in its dependency tree due to clap:

├── clap v4.1.11
│   ├── bitflags v2.0.2
│   ├── clap_derive v4.1.9 (proc-macro)
│   │   ├── heck v0.4.1
│   │   ├── proc-macro-error v1.0.4
│   │   │   ├── proc-macro-error-attr v1.0.4 (proc-macro)
│   │   │   │   ├── proc-macro2 v1.0.52 (*)
│   │   │   │   └── quote v1.0.26 (*)
│   │   │   │   [build-dependencies]
│   │   │   │   └── version_check v0.9.4
│   │   │   ├── proc-macro2 v1.0.52 (*)
│   │   │   ├── quote v1.0.26 (*)
│   │   │   └── syn v1.0.109 (*)
│   │   │   [build-dependencies]
│   │   │   └── version_check v0.9.4
│   │   ├── proc-macro2 v1.0.52 (*)
│   │   ├── quote v1.0.26 (*)
│   │   └── syn v1.0.109 (*)
│   ├── clap_lex v0.3.3
│   │   └── os_str_bytes v6.5.0
│   ├── is-terminal v0.4.5
│   │   ├── io-lifetimes v1.0.9
│   │   │   └── libc v0.2.140
│   │   └── rustix v0.36.11
│   │       ├── bitflags v1.3.2
│   │       ├── io-lifetimes v1.0.9 (*)
│   │       ├── libc v0.2.140
│   │       └── linux-raw-sys v0.1.4
│   ├── once_cell v1.17.1
│   ├── strsim v0.10.0
│   └── termcolor v1.2.0

I have no idea why but sometimes rustix will just randomly trigger a rebuild.

   Compiling rustix v0.36.11
    Checking is-terminal v0.4.5
    Checking clap v4.1.11
    Checking inferno v0.11.15
    Checking shotover-proxy v0.1.9 (/home/rukai2/Projects/Crates/shotover/shotover-proxy/shotover-proxy)
    Checking test-helpers v0.1.0 (/home/rukai2/Projects/Crates/shotover/shotover-proxy/test-helpers)
    Finished dev [unoptimized + debuginfo] target(s) in 6.51s

I've never seen this for any other dependency before.

If I observe anything else that might explain whats going on I'll add it to this issue.

The text was updated successfully, but these errors were encountered:

sunfishcode · 2023-03-28T23:27:18Z

Are you using an IDE, such as IntelliJ or another? Are you using cargo watch or similar tool which run commands automatically?

rukai · 2023-03-28T23:29:32Z

I have vs code + rust analyzer open and bacon open at the same time.

sunfishcode · 2023-03-29T00:42:40Z

Is it possible that bacon is using a different rust compiler, or different rust flags, than vs code? Rustix's build.rs script autodetects some features from the rust compiler, so if it appears the rust compiler is changing, it may be triggering the rebuilds.

Some additional context is: rustix has recently had issues with build.rs not getting run when it needs to, which led to #544, #563, and #565, which seem to have fixed that issue, but now seem to be causing build.rs to run more often than it needs to. I don't yet understand how and when build.rs gets run in many situations, and don't yet know why this issue seems to be afflicting rustix when it doesn't seem to affect other crates that do rustc feature detection.

rukai · 2023-03-29T01:46:49Z

Looking through the rust-analyzer vs code config I saw:

Which corresponds to

rustix/build.rs

Line 153 in 1a9d8da

println!("cargo:rerun-if-env-changed=RUSTC_WRAPPER");

But also in that case wouldnt

rustix/build.rs

Line 154 in 1a9d8da

println!("cargo:rerun-if-env-changed=PROFILE");

cause a rebuild every time I switch between release and debug? Which I've observed is not the case.

Cargo must have its own internal logic for skipping rebuilds before deferring to the build.rs rerun-if logic?
Or maybe every env var that is usually available when the build.rs actually runs is not available for the rerun-if checks?

rukai · 2023-03-29T01:52:37Z

https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorerun-if-env-changedname

Note that the environment variables here are intended for global environment variables like CC and such, it is not necessary to use this for environment variables like TARGET that Cargo sets.

Oh this looks useful: https://doc.rust-lang.org/cargo/faq.html#why-is-cargo-rebuilding-my-code

rukai · 2023-03-29T02:13:29Z

I discovered that cargo recently had reasons for rebuilds added to its output when --verbose is used.

Through doing this I was able to catch RUSTC_WRAPPER as the cause:

Now that we appear to have a known-working fix for #526, we can investigate relaxing the fix. If #526 reappears, we now have a known-working state we can quickly revert to. Reports in #562 and #575 are that having rerun-if-env-changed=RUSTC and rerun-if-env-changed=RUSTC_WRAPPER are triggering spurious rebuilds in some situations, so remove these. In theory, cargo should already be aware of these environment variables. In practice, there were reports of build failures which had the appearance of using an older version of rustc with build.rs output from a newer version of rustc, and it wasn't clear what was happening. But, it's possible that #563 fixes the issue properly. So let's try removing these rerun-if-env-changed lines, and see if the problem resurfaces. And, rerun-if-env-changed=TARGET is explicitly documented as being unnecessary [here]. [here]: https://doc.rust-lang.org/cargo/reference/build-scripts.html

VorpalBlade · 2023-03-31T18:58:02Z

I'm not sure if this is the same issue, as I get a slightly different info with cargo build --verbose:

Dirty rustix v0.37.4: the dependency build_script_build was rebuilt (1680288365.394171857s, 7m 15s after last build at 1680287930.340354402s)

Unfortunately that doesn't tell which env var it is. I too am using VSCode with rust-analyzer.

theHausdorffMetric · 2023-04-12T06:41:03Z

I'm not sure if this is the same issue, as I get a slightly different info with cargo build --verbose:
Dirty rustix v0.37.4: the dependency build_script_build was rebuilt (1680288365.394171857s, 7m 15s after last build at 1680287930.340354402s)
Unfortunately that doesn't tell which env var it is. I too am using VSCode with rust-analyzer.

Same issue here (rustix keeps rebuilding) with same cargo verbose output, but using helix editor with rust-analyzer.

rudyhb · 2023-04-13T19:30:35Z

Same issue for me with CLion + rust plugin

sunfishcode · 2023-04-13T19:34:22Z

Could people still seeing this issue please include what version of rustix they're seeing it in?

I am currently unable to reproduce this, so any extra information you might be able to provide about the circumstances you see this in could be helpful. Thanks!

rukai · 2023-04-13T23:19:18Z

I havent seen it in a while, I'll report back if I do see it again.

rudyhb · 2023-04-14T01:44:49Z

sorry about that, issue seems to be fixed after updating from v0.36.11 to v0.37.11

theHausdorffMetric · 2023-04-14T07:22:34Z

sorry about that, issue seems to be fixed after updating from v0.36.11 to v0.37.11

Many thx - preliminary testing v0.37.11 in my case (helix editor with rust-analyzer) no longer shows the problem :-)

rukai · 2023-05-02T03:05:45Z

I havent seen it in a while, I'll report back if I do see it again.

I havent seen the problem since then so I'm going to go ahead and close the issue.
Thanks for resolving it!

Now that we appear to have a known-working fix for #526, we can investigate relaxing the fix. If #526 reappears, we now have a known-working state we can quickly revert to. Reports in #562 and #575 are that having rerun-if-env-changed=RUSTC and rerun-if-env-changed=RUSTC_WRAPPER are triggering spurious rebuilds in some situations, so remove these. In theory, cargo should already be aware of these environment variables. In practice, there were reports of build failures which had the appearance of using an older version of rustc with build.rs output from a newer version of rustc, and it wasn't clear what was happening. But, it's possible that #563 fixes the issue properly. So let's try removing these rerun-if-env-changed lines, and see if the problem resurfaces. And, rerun-if-env-changed=TARGET is explicitly documented as being unnecessary [here]. [here]: https://doc.rust-lang.org/cargo/reference/build-scripts.html

This transitevely bumps rustix so it won't re-compile due to bytecodealliance/rustix#575 Change-Id: I616b6cb0d96bb9c4bf22c850253caefd6a7b273c Reviewed-on: https://gerrit.readyset.name/c/readyset/+/5515 Tested-by: Buildkite CI Reviewed-by: Aspen Smith <aspen@readyset.io>

Now that we appear to have a known-working fix for #526, we can investigate relaxing the fix. If #526 reappears, we now have a known-working state we can quickly revert to. Reports in #562 and #575 are that having rerun-if-env-changed=RUSTC and rerun-if-env-changed=RUSTC_WRAPPER are triggering spurious rebuilds in some situations, so remove these. In theory, cargo should already be aware of these environment variables. In practice, there were reports of build failures which had the appearance of using an older version of rustc with build.rs output from a newer version of rustc, and it wasn't clear what was happening. But, it's possible that #563 fixes the issue properly. So let's try removing these rerun-if-env-changed lines, and see if the problem resurfaces. And, rerun-if-env-changed=TARGET is explicitly documented as being unnecessary [here]. [here]: https://doc.rust-lang.org/cargo/reference/build-scripts.html

sunfishcode mentioned this issue Mar 29, 2023

Remove some rerun-if-env-changed lines from build.rs. #576

Merged

rukai closed this as completed May 2, 2023

sunfishcode mentioned this issue May 19, 2023

Remove some rerun-if-env-changed lines from build.rs. (#576) #674

Merged

th1000s mentioned this issue Jul 17, 2023

Bump clap to v4.3.14 and regenerate Cargo.lock dandavison/delta#1472

Merged

lukoktonos mentioned this issue Jul 21, 2023

bump terminal_size to 0.2.6 clap-rs/clap#5036

Closed

timvisee mentioned this issue Nov 6, 2023

Attempt to reduce build cache invalidation qdrant/qdrant#2935

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rustix keeps rebuilding in my dependency tree. #575

rustix keeps rebuilding in my dependency tree. #575

rukai commented Mar 28, 2023

sunfishcode commented Mar 28, 2023

rukai commented Mar 28, 2023 •

edited

Loading

sunfishcode commented Mar 29, 2023

rukai commented Mar 29, 2023 •

edited

Loading

rukai commented Mar 29, 2023

rukai commented Mar 29, 2023

VorpalBlade commented Mar 31, 2023

theHausdorffMetric commented Apr 12, 2023 •

edited

Loading

rudyhb commented Apr 13, 2023

sunfishcode commented Apr 13, 2023

rukai commented Apr 13, 2023

rudyhb commented Apr 14, 2023

theHausdorffMetric commented Apr 14, 2023

rukai commented May 2, 2023

rustix keeps rebuilding in my dependency tree. #575

rustix keeps rebuilding in my dependency tree. #575

Comments

rukai commented Mar 28, 2023

sunfishcode commented Mar 28, 2023

rukai commented Mar 28, 2023 • edited Loading

sunfishcode commented Mar 29, 2023

rukai commented Mar 29, 2023 • edited Loading

rukai commented Mar 29, 2023

rukai commented Mar 29, 2023

VorpalBlade commented Mar 31, 2023

theHausdorffMetric commented Apr 12, 2023 • edited Loading

rudyhb commented Apr 13, 2023

sunfishcode commented Apr 13, 2023

rukai commented Apr 13, 2023

rudyhb commented Apr 14, 2023

theHausdorffMetric commented Apr 14, 2023

rukai commented May 2, 2023

rukai commented Mar 28, 2023 •

edited

Loading

rukai commented Mar 29, 2023 •

edited

Loading

theHausdorffMetric commented Apr 12, 2023 •

edited

Loading