Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for build plan generation (--build-plan) #5579

Open
jonas-schievink opened this issue May 27, 2018 · 11 comments
Open

Tracking issue for build plan generation (--build-plan) #5579

jonas-schievink opened this issue May 27, 2018 · 11 comments

Comments

@jonas-schievink
Copy link
Member

@jonas-schievink jonas-schievink commented May 27, 2018

Implemented in #5301, Cargo gained the ability to produce a JSON build plan containing a list of commands that need to be executed to build a target.

The feature is currently unstable and requires a nightly Cargo release as well as the -Zunstable-options command line argument. This issue tracks its eventual stabilization.

This feature was originally requested in #3815 to facilitate Cargo's integration into other build systems.

@jonas-schievink
Copy link
Member Author

@jonas-schievink jonas-schievink commented May 28, 2018

A perhaps non-obvious use case of this: I was able to reimplement the compile-fail test functionality provided by compiletest-rs using the build plan to figure out how to compile an integration test: https://github.com/jonas-schievink/compile-fail

The benefit of this approach is that asking Cargo how to build a test is much more robust than building the rustc command line manually (essentially guessing how dependencies should be linked). This should fix practically every bug where compiletest-rs would break with an error like "multiple matching crates found".

Loading

@saleemrashid
Copy link

@saleemrashid saleemrashid commented Jun 26, 2018

One complication is that the downstream build system needs to know how to use build scripts. Otherwise, the build plan will execute the build script, without passing the --cfg options to subsequent rustc invocations.

Loading

@saleemrashid
Copy link

@saleemrashid saleemrashid commented Jun 26, 2018

Also, including the path to the dep-info file would be useful, as it is not as simple as changing the file extension to .d. Libraries prefix the artifacts with lib, but not the dep-info files.

Loading

@luser
Copy link
Contributor

@luser luser commented Jul 12, 2018

I don't know if there's a useful way to associate issues with a tracking issue in GitHub, but I filed: #5719

Loading

@hobofan
Copy link

@hobofan hobofan commented Nov 13, 2018

I just learned about the --build-plan as I was looking to integrate it into a future version of cargo-nono.

One thing that was really surprising to me, that there hasn't been any discussions about the implications of build.rs output altering the build plan as far as I can tell. I feel like that if that factor is ignored it would give either give consumers of the build plan a false sense of security that the produced build plan is static and always acurate, or leave a significant feature with many crates using it up for reeimplementation by the build-plan consumers.

To me it seems like that is something at least worth discussing before this feature will be stabilized.

Loading

@denzp
Copy link

@denzp denzp commented Nov 13, 2018

@hobofan Indeed, it's impossible to predict what build script will do. It can create some files, alter the rustc environment, or add link flags.

For my use case (experiments with advanced Docker building and caching), I've made a small binary that handles build script output with cargo's cargo::core::compiler::BuildOutput.

Probably build-plan could be improved in a way that build script invocation should be somehow grouped with the target crate's rustc invocation.

Loading

@LukasKalbertodt
Copy link
Member

@LukasKalbertodt LukasKalbertodt commented Jun 5, 2019

I'm not sure if it fits this issue, but using build plans broke for auto_impl. We execute cargo build -Z unstable-options --build-plan to get the build plan. Unfortunately between rustc 1.36.0-nightly (7158ed9cb 2019-05-15) and rustc 1.36.0-nightly (7d5aa4332 2019-05-16) that command started removing .rlib files from target/debug/deps.

To reproduce, just create a new library project, add an dependency, compile with cargo build, take a look at target/debug/deps to make sure it contains .rlib files and execute cargo build -Z unstable-options --build-plan. Now the files are gone.

Is that intended?

Loading

@jonas-schievink
Copy link
Member Author

@jonas-schievink jonas-schievink commented Jun 5, 2019

Yeah, the build-plan integration test calls cargo --build-plan too and broke in a weird way due to this. I've opened #6954 a while ago because of that.

Loading

@CAD97
Copy link
Contributor

@CAD97 CAD97 commented Oct 28, 2019

--build-plan causes everything to need recompiling, even if it was fresh before.

Example
PS D:\usr\Documents\Code\Rust\crafting-compilers> cargo +nightly clean
PS D:\usr\Documents\Code\Rust\crafting-compilers> cargo +nightly build -Z unstable-options  
   Compiling memchr v2.2.1
   Compiling winapi-x86_64-pc-windows-gnu v0.4.0
   Compiling lazy_static v1.4.0
   Compiling winapi v0.3.8
   Compiling proc-macro2 v1.0.5
   Compiling unicode-xid v0.2.0
   Compiling syn v1.0.5
   Compiling autocfg v0.1.6
   Compiling regex-syntax v0.6.12
   Compiling cfg-if v0.1.10                                                                                                                                              
   Compiling libc v0.2.62                                                                                                                                                
   Compiling log v0.4.8                                                                                                                                                  
   Compiling ucd-trie v0.1.2
   Compiling serde v1.0.101                                                                                                                                              
   Compiling unic-char-range v0.9.0                                                                                                                                      
   Compiling ryu v1.0.0
   Compiling unic-common v0.9.0
   Compiling fnv v1.0.6
   Compiling maplit v1.0.2                                                                                                                                               
   Compiling deunicode v0.4.3
   Compiling itoa v0.4.4
   Compiling humansize v1.1.0                                                                                                                                            
   Compiling percent-encoding v2.1.0
   Compiling unicode-segmentation v1.3.0
   Compiling glob v0.3.0
   Compiling thread_local v0.3.6
   Compiling crossbeam-utils v0.6.6
   Compiling num-traits v0.2.8                                                                                                                                           
   Compiling num-integer v0.1.41
   Compiling pest v2.1.2                                                                                                                                                 
   Compiling unic-char-property v0.9.0
   Compiling unic-ucd-version v0.9.0
   Compiling slug v0.1.4
   Compiling heck v0.3.1
   Compiling crossbeam-channel v0.3.9                                                                                                                                    
   Compiling unic-ucd-segment v0.9.0
   Compiling quote v1.0.2
   Compiling pest_meta v2.1.2
   Compiling bstr v0.2.8
   Compiling parse-zoneinfo v0.2.0
   Compiling globset v0.4.4
   Compiling winapi-util v0.1.2
   Compiling time v0.1.42
   Compiling same-file v1.0.5
   Compiling chrono v0.4.9
   Compiling walkdir v2.2.9
   Compiling chrono-tz v0.5.1
   Compiling ignore v0.4.10
   Compiling pest_generator v2.1.1
   Compiling globwalk v0.7.1
   Compiling serde_derive v1.0.101
   Compiling pest_derive v2.1.0
   Compiling serde_json v1.0.41
   Compiling toml v0.5.3
   Compiling tera v1.0.0-beta.18
   Compiling tinyc_grammar v0.1.0 (D:\usr\Documents\Code\Rust\crafting-compilers\crates\grammar)
   Compiling tinyc_lexer v0.1.0 (D:\usr\Documents\Code\Rust\crafting-compilers\crates\lexer)
    Finished dev [unoptimized + debuginfo] target(s) in 1m 05s
PS D:\usr\Documents\Code\Rust\crafting-compilers> cargo +nightly build -Z unstable-options
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
PS D:\usr\Documents\Code\Rust\crafting-compilers> cargo +nightly build -Z unstable-options --build-plan > $null 
PS D:\usr\Documents\Code\Rust\crafting-compilers> cargo +nightly build -Z unstable-options
   Compiling lazy_static v1.4.0
   Compiling unicode-xid v0.2.0
   Compiling regex-syntax v0.6.12
   Compiling autocfg v0.1.6
   Compiling cfg-if v0.1.10
   Compiling ucd-trie v0.1.2
   Compiling unic-char-range v0.9.0
   Compiling maplit v1.0.2
   Compiling fnv v1.0.6
   Compiling unic-common v0.9.0
   Compiling deunicode v0.4.3
   Compiling itoa v0.4.4
   Compiling percent-encoding v2.1.0
   Compiling humansize v1.1.0
   Compiling unicode-segmentation v1.3.0
   Compiling glob v0.3.0
   Compiling thread_local v0.3.6
   Compiling crossbeam-utils v0.6.6
   Compiling unic-char-property v0.9.0
   Compiling pest v2.1.2                                                                                                                                                 
   Compiling unic-ucd-version v0.9.0
   Compiling slug v0.1.4
   Compiling memchr v2.2.1
   Compiling winapi-x86_64-pc-windows-gnu v0.4.0
   Compiling proc-macro2 v1.0.5
   Compiling log v0.4.8
   Compiling libc v0.2.62
   Compiling ryu v1.0.0
   Compiling heck v0.3.1
   Compiling num-traits v0.2.8
   Compiling num-integer v0.1.41
   Compiling unic-ucd-segment v0.9.0
   Compiling crossbeam-channel v0.3.9
   Compiling winapi v0.3.8
   Compiling aho-corasick v0.7.6                                                                                                                                         
   Compiling bstr v0.2.8                                                                                                                                                 
   Compiling quote v1.0.2                                                                                                                                                
   Compiling pest_meta v2.1.2
   Compiling unic-segment v0.9.0
   Compiling syn v1.0.5                                                                                                                                                  
   Compiling regex v1.3.1
   Compiling parse-zoneinfo v0.2.0
   Compiling globset v0.4.4
   Compiling winapi-util v0.1.2
   Compiling time v0.1.42
   Compiling same-file v1.0.5
   Compiling walkdir v2.2.9
   Compiling chrono v0.4.9
   Compiling ignore v0.4.10
   Compiling chrono-tz v0.5.1
   Compiling pest_generator v2.1.1
   Compiling globwalk v0.7.1
   Compiling serde_derive v1.0.101
   Compiling pest_derive v2.1.0
   Compiling serde v1.0.101
   Compiling serde_json v1.0.41
   Compiling toml v0.5.3
   Compiling tera v1.0.0-beta.18
   Compiling tinyc_grammar v0.1.0 (D:\usr\Documents\Code\Rust\crafting-compilers\crates\grammar)
   Compiling tinyc_lexer v0.1.0 (D:\usr\Documents\Code\Rust\crafting-compilers\crates\lexer)
    Finished dev [unoptimized + debuginfo] target(s) in 1m 01s

Is this intended, a bug, or an unfortunate requirement? This makes using --build-plan for IDE integrations (IntelliJ-Rust, rust-analyzer) unexpectedly expensive.

Loading

@vlad20012
Copy link
Member

@vlad20012 vlad20012 commented Oct 28, 2019

I've reproduced @CAD97's issue. Could someone please investigate it? It's really blocking for IntelliJ-Rust env!("OUT_DIR") support

Loading

Undin added a commit to intellij-rust/intellij-rust that referenced this issue Oct 28, 2019
Found out that `cargo build --build-plan` force next `build` command to recompile whole project (including dependencies).
See rust-lang/cargo#5579 (comment).
At this moment it is part of refresh, i.e. it is invoked on each `Cargo.toml` change, project reopening, etc. So this command is invoked quite frequently.
It can be rather unexpected for large project to make full compilation after each project reopening
Undin added a commit to intellij-rust/intellij-rust that referenced this issue Oct 29, 2019
…ault

Recently supported name resolution for included files uses `cargo build --build-plan` to fetch value of `OUT_DIR` environment variable (see #4542).
Found out that `cargo build --build-plan` force next `build` command to recompile whole project (including dependencies).
See rust-lang/cargo#5579 (comment).
At this moment it is part of refresh, i.e. it is invoked on each `Cargo.toml` change, project reopening, etc. So this command is invoked quite frequently.
It can be rather unexpected for large project to make full compilation after each project reopening

So we decided to disable it by default until it will be fixed from cargo side or we'll find better way to get `OUT_DIR` value
bors bot added a commit to intellij-rust/intellij-rust that referenced this issue Oct 29, 2019
4573: MACRO: disable name resolution for include with env("OUT_DIR") by default r=Undin a=Undin

Recently supported name resolution for included files uses `cargo build --build-plan` to fetch value of `OUT_DIR` environment variable (see #4542).
Found out that `cargo build --build-plan` force next `build` command to recompile whole project (including dependencies).
See rust-lang/cargo#5579 (comment).
At this moment it is part of refresh, i.e. it is invoked on each `Cargo.toml` change, project reopening, etc. So this command is invoked quite frequently.
It can be rather unexpected for large project to make full compilation after each project reopening

So we decided to disable it by default until it will be fixed from cargo side or we'll find better way to get `OUT_DIR` value


Co-authored-by: Arseniy Pendryak <a.pendryak@yandex.ru>
@alexcrichton
Copy link
Member

@alexcrichton alexcrichton commented Dec 2, 2019

Note that the current conclusion in #7614 is to delete build plan support. If you're currently relying on build plans, please be sure to check that issue out and comment if you have thoughts!

Loading

@ehuss ehuss added this to Unstable, no backers in Roadmap Dec 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Roadmap
  
Unstable, no backers
Linked pull requests

Successfully merging a pull request may close this issue.

None yet