MACRO: support `env!("OUT_DIR")` in include macro calls by Undin · Pull Request #4542 · intellij-rust/intellij-rust

Undin · 2019-10-18T10:57:42Z

Under the hood, it uses output of cargo build --build-plan command to get value of OUT_DIR variable

Fixes #1908

Undin · 2019-10-20T21:53:27Z

I've filed a separate issue #4546 related to the corresponding user notification about the absence of expected generated files

src/main/kotlin/org/rust/lang/core/psi/ext/RsMacroCall.kt

This value need to support code included via include macro calls like `include!(concat!(env!("OUT_DIR"), "/bindings.rs"))`. At this moment this info extracted from output of `cargo build --build-plan` command.

…ROP_UNLOADED_PSI` It should invalidate caches when file is generated by compiler in target dir

Undin · 2019-10-25T09:54:13Z

@vlad20012 I've fixed all discussed issues

vlad20012 · 2019-10-25T11:29:19Z

bors r+

4542: MACRO: support `env!("OUT_DIR")` in include macro calls r=vlad20012 a=Undin Under the hood, it uses output of `cargo build --build-plan` command to get value of `OUT_DIR` variable ![2019-10-18 14 03 23](https://user-images.githubusercontent.com/2539310/67089379-1cb9fd80-f1b0-11e9-95c3-1fec12f9095d.gif) Fixes #1908 Co-authored-by: Arseniy Pendryak <a.pendryak@yandex.ru>

bors · 2019-10-25T11:55:37Z

Build succeeded

CAD97 · 2019-10-28T01:48:42Z

Thought I should drop this here:

I've requested a way to get OUT_DIR from cargo metadata because cargo build --build-plan always causes a full rebuild after.

CAD97 · 2019-10-28T01:52:04Z

src/main/kotlin/org/rust/cargo/toolchain/Cargo.kt

+        val additionalArgs = mutableListOf("-Z", "unstable-options", "--all-targets", "--build-plan")
+        // Hack to make cargo think that current channel is nightly because we need unstable `--build-plan` option here
+        val envs = EnvironmentVariablesData.create(mapOf(
+            RUSTC_BOOTSTRAP to "1"


This means that this functionality can break at any moment.

We are closely monitoring changes in the rust toolchain and ready for breaking changes in the build-plan or other nightly features that we use. We'd want a more explicit way to unlock all nightly command-line options on the stable toolchain but now there is only "RUSTC_BOOTSTRAP" magic.

vlad20012 · 2019-10-28T13:10:19Z

@CAD97 thanks for the comment! Now we're going to disable the feature because of this. (the feature has not yet been released)

bjorn3 · 2019-10-28T13:12:30Z

@vlad20012 Maybe enable it only when the user is actually using a nightly compiler. Or do you want to not diverge the supported features between stable and nightly users?

vlad20012 · 2019-10-28T13:20:02Z

@bjorn3 Umm, I'm about the issue that build-plan invocation forces cargo to make full rebuild on the next build. I don't see differences between stable and nightly compiler in this case. But we'll make some option to enable this feature back

…ault Recently supported name resolution for included files uses `cargo build --build-plan` to fetch value of `OUT_DIR` environment variable (see #4542). Found out that `cargo build --build-plan` force next `build` command to recompile whole project (including dependencies). See rust-lang/cargo#5579 (comment). At this moment it is part of refresh, i.e. it is invoked on each `Cargo.toml` change, project reopening, etc. So this command is invoked quite frequently. It can be rather unexpected for large project to make full compilation after each project reopening So we decided to disable it by default until it will be fixed from cargo side or we'll find better way to get `OUT_DIR` value

4573: MACRO: disable name resolution for include with env("OUT_DIR") by default r=Undin a=Undin Recently supported name resolution for included files uses `cargo build --build-plan` to fetch value of `OUT_DIR` environment variable (see #4542). Found out that `cargo build --build-plan` force next `build` command to recompile whole project (including dependencies). See rust-lang/cargo#5579 (comment). At this moment it is part of refresh, i.e. it is invoked on each `Cargo.toml` change, project reopening, etc. So this command is invoked quite frequently. It can be rather unexpected for large project to make full compilation after each project reopening So we decided to disable it by default until it will be fixed from cargo side or we'll find better way to get `OUT_DIR` value Co-authored-by: Arseniy Pendryak <a.pendryak@yandex.ru>

8012: CARGO: improve work with generated sources r=vlad20012 a=Undin In Rust ecosystem, it's quite common to generate some source via build scripts during compilation. Usually, such generated sources are included into project via `include!` macro. The plugin can work with such sources but the corresponding support is currently under `org.rust.cargo.evaluate.build.scripts` [experimental feature](https://plugins.jetbrains.com/plugin/8182-rust/docs/rust-faq.html#experimental-features). See #1908 for more context and details Since #4542, the plugin added all `out` directories of all packages as source roots into project content root entry. And it worked in general, but this approach has some disadvantages: - each `out` directory is displayed in `Project Structure` settings. If a project uses many libraries that produce a lot of own `out` directories, `Project Structure` becomes too large and complicated for manual editing - it doesn't work if `out` directory is located outside of the project directory (for example, because of the custom path of `target` directory) The new approach just provides all `out` directories via `IndexableSetContributor` API to make IDE index generated sources that solve all described issues Fixes #6808 Fixes #6844 Fixes #6998 Fixes #7583 Fixes #7934 changelog: Improve work with generated sources provided via `org.rust.cargo.evaluate.build.scripts` [experimental feature](https://plugins.jetbrains.com/plugin/8182-rust/docs/rust-faq.html#experimental-features). Now the plugin doesn't pollute `Project Structure` view with entries for generated `out` directories. Also, the custom paths of target directories are also supported Co-authored-by: Arseniy Pendryak <a.pendryak@yandex.ru>

Undin added the feature label Oct 18, 2019

Undin assigned vlad20012 Oct 18, 2019

Undin force-pushed the undin/env-out-dir branch from 42e2eed to 43311cb Compare October 18, 2019 12:28

jrmuizel mentioned this pull request Oct 18, 2019

Handle include!(concat!(env!("OUT_DIR"), "/bindings.rs")); #1908

Closed

3 tasks

Undin force-pushed the undin/env-out-dir branch from 43311cb to 4115087 Compare October 19, 2019 22:40

Undin mentioned this pull request Oct 20, 2019

Inform user to build dependencies/project to create sources generated by build scripts #4546

Closed

vlad20012 reviewed Oct 22, 2019

View reviewed changes

src/main/kotlin/org/rust/lang/core/psi/ext/RsMacroCall.kt Outdated Show resolved Hide resolved

Undin force-pushed the undin/env-out-dir branch from 4115087 to 0e0e374 Compare October 24, 2019 08:06

Undin mentioned this pull request Oct 24, 2019

Fix NPE in RsGeneratedSourceFilter on 2019.3 #4560

Merged

Undin added 2 commits October 25, 2019 12:42

CARGO: Fetch value of OUT_DIR env variable for targets

6658bfc

This value need to support code included via include macro calls like `include!(concat!(env!("OUT_DIR"), "/bindings.rs"))`. At this moment this info extracted from output of `cargo build --build-plan` command.

Mark out dir as source

80438e8

Undin force-pushed the undin/env-out-dir branch from 0e0e374 to 21862de Compare October 25, 2019 09:49

Undin added 6 commits October 25, 2019 12:50

GRAM&STUB: add special grammar rule for env macro

0b0252a

RES&MACRO: support env("OUT_DIR") in include macro call

c106727

CARGO: correctly determine cargo project and package by target.outDir

926747b

PSI: increase structure modification counter on `PsiTreeChangeEvent.P…

a5a2287

…ROP_UNLOADED_PSI` It should invalidate caches when file is generated by compiler in target dir

Mark code in out dir as generated and don't allow to modify them

11b0251

T: test with real Cargo

f185c10

Undin force-pushed the undin/env-out-dir branch from 21862de to f185c10 Compare October 25, 2019 09:51

Undin added this to the v109 milestone Oct 25, 2019

bors bot merged commit f185c10 into master Oct 25, 2019

bors bot deleted the undin/env-out-dir branch October 25, 2019 11:55

Undin mentioned this pull request Oct 25, 2019

Code autocompletion not working for crates using C libraries #3257

Closed

This was referenced Oct 28, 2019

env!("OUT_DIR") macro rust-lang/rust-analyzer#1967

Closed

Tracking issue for build plan generation (--build-plan) rust-lang/cargo#5579

Closed

CAD97 reviewed Oct 28, 2019

View reviewed changes

Undin mentioned this pull request Oct 29, 2019

MACRO: disable name resolution for include with env("OUT_DIR") by default #4573

Merged

Undin mentioned this pull request Nov 2, 2019

MACRO: prevent stackoverflow while name resolution for include macro #4584

Merged

Undin mentioned this pull request Oct 27, 2021

CARGO: improve work with generated sources #8012

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MACRO: support `env!("OUT_DIR")` in include macro calls#4542

MACRO: support `env!("OUT_DIR")` in include macro calls#4542
bors[bot] merged 8 commits intomasterfrom
undin/env-out-dir

Undin commented Oct 18, 2019 •

edited

Loading

Uh oh!

Undin commented Oct 20, 2019

Uh oh!

Uh oh!

Undin commented Oct 25, 2019

Uh oh!

vlad20012 commented Oct 25, 2019

Uh oh!

bors bot commented Oct 25, 2019

Uh oh!

CAD97 commented Oct 28, 2019 •

edited

Loading

Uh oh!

CAD97 Oct 28, 2019

Uh oh!

bjorn3 Oct 28, 2019

Uh oh!

vlad20012 Oct 28, 2019

Uh oh!

vlad20012 commented Oct 28, 2019 •

edited

Loading

Uh oh!

bjorn3 commented Oct 28, 2019 •

edited

Loading

Uh oh!

vlad20012 commented Oct 28, 2019 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Undin commented Oct 18, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Undin commented Oct 20, 2019

Uh oh!

Uh oh!

Undin commented Oct 25, 2019

Uh oh!

vlad20012 commented Oct 25, 2019

Uh oh!

bors bot commented Oct 25, 2019

Build succeeded

Uh oh!

CAD97 commented Oct 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CAD97 Oct 28, 2019

Choose a reason for hiding this comment

Uh oh!

bjorn3 Oct 28, 2019

Choose a reason for hiding this comment

Uh oh!

vlad20012 Oct 28, 2019

Choose a reason for hiding this comment

Uh oh!

vlad20012 commented Oct 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bjorn3 commented Oct 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vlad20012 commented Oct 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Undin commented Oct 18, 2019 •

edited

Loading

CAD97 commented Oct 28, 2019 •

edited

Loading

vlad20012 commented Oct 28, 2019 •

edited

Loading

bjorn3 commented Oct 28, 2019 •

edited

Loading

vlad20012 commented Oct 28, 2019 •

edited

Loading