Support for JetBrains IDEs

[mahkoh]

Without Cargo.toml files, some significant dev exp features are inaccessible. This includes any kind of support for JetBrains IDEs.

This is not a request for crates to be built with cargo.

It might be possible to generate Cargo.toml files automatically. I believe that some files are already being generated for rust-analyzer support. If not, then it might be necessary to maintain such files manually.

[bjorn3]

AFAIK IntelliJ doesn't have support for a custom sysroot at all. Rust-for-linux uses a custom sysroot. It also doesn't have a way to point directly to arbitrary compiled proc-macros, unlike rust-analyzer. IntelliJ works best with projects that use cargo with a standard project layout. Even the rust compiler itself, which uses cargo, isn't supported by IntelliJ due to the unusual project layout.

Generating Cargo.toml files would be non-trivial as multiple drivers (which need a separate Cargo.toml file) may be present in a single directory. Rust-analyzer supports this just fine with rust-project.json, but cargo doesn't support this.

Cargo.toml files being present may also cause people to attempt to build using cargo (whether intentionally or due to their IDE automatically running cargo build) which does not work. Not the least because rust-for-linux depends on a bunch of env vars and --cfg arguments which can't be set using Cargo.toml. Without those set a lot of code in the kernel crate is configured out, which would likely also break IDE functionality in IntelliJ.

[ojeda]

If not, then it might be necessary to maintain such files manually.

No, we can't really commit and maintain Cargo.toml files manually everywhere just for a single IDE.

Generating files on the fly is possible, but even if that happened to work (which as @bjorn3 says, it wouldn't, at least not easily), there is already rust-project.json for that (which is better suited for that purpose too).

JetBrains should support non-Cargo Rust projects, and ideally just add support for rust-project.json, since I would expect that is what most non-Cargo Rust projects already use, and it is harder to justify targets/features for individual IDEs.

[ojeda]

@mahkoh Perhaps we could reuse this issue for something like "Support for JetBrains IDEs"? i.e. to track support in general, in whatever form it comes eventually.

[mahkoh]

Sure, that is the primary issue I have.

The issues you mentioned so far sound reasonable and I'm currently out of the office so I cannot investigate them myself.

[ojeda]

Sounds good, thanks!

[mahkoh]

It might be possible to generate Cargo.toml files automatically.

I've written such a tool: https://github.com/mahkoh/rust-project-to-cargo

My goal was to achieve the following:

• All of the usual IDE features work across the entire source tree: Expand macros (including proc macros), go to definition, find usages (including for proc macros), rename.
• cargo clippy runs to completion without errors.

And this mostly works now. However, I had to make some small changes to existing files. You can see all of the required changes plus checked-in Cargo.toml files in this pull request: mahkoh#2

The checked-in Cargo.toml files will not work for you since they contain absolute paths. Regenerate them with the tool if you want to test them.

The small changes amount to the following:

1. Some crates depend on an ffi crate but this crate is not declared in rust-project.toml. I'm not sure if/how this works with rust analyzer but I've just added this crate to generate_rust_analyzer.py.
2. Some crates do not have a #![no_std] annotation. This seems to work during normal compilation since the makefiles passes --sysroot=/dev/null but doing so with Cargo is not viable. I've added #![no_std] to these files to fix this.

Additionally, there are some annoyances:

1. Rustc emits warnings that some features are stable. During normal compilation, these features seem to be configured out but those configure settings don't seem to be included in rust-project.toml.
2. rust-project.toml seems to always include all crates, including those that are configured out. This causes errors while running clippy because those crates depend on features in other crates that are configured out. E.g. nova_core depends on the kernel/firmware module.

So, the changes required to make this seamless would be

1. Declare the ffi crate.
2. Require that all crates have a #![no_std] annotation.
3. Ensure that the cfgs included in rust-project.toml actually correspond to the cfgs used during compilation.
4. Don't include configured-out crates in rust-project.toml.

[bjorn3]

Some crates do not have a #![no_std] annotation. This seems to work during normal compilation since the makefiles passes --sysroot=/dev/null but doing so with Cargo is not viable. I've added #![no_std] to these files to fix this.

The build system passes -Zcrate-attr=no_std which adds #![no_std] after the crate root is parsed. It also passes a bunch of other -Zcrate-attr for eg unstable features.

Rustc emits warnings that some features are stable. During normal compilation, these features seem to be configured out but those configure settings don't seem to be included in rust-project.toml.

The build system passes a rustc flag to disable those warnings.

Don't include configured-out crates in rust-project.toml.

This would break rust-analyzer on configured-out crates, breaking eg project wide refactors.

cargo clippy runs to completion without errors.

We already have a make command for clippy.

[mahkoh]

The build system passes -Zcrate-attr=no_std which adds #![no_std] after the crate root is parsed. It also passes a bunch of other -Zcrate-attr for eg unstable features.

[...]

The build system passes a rustc flag to disable those warnings.

It would be easy to include these flags via .cargo/config.toml in the generated files but I see no way to figure out what those flags are from the rust-project.toml. I guess the user would have to manually maintain them?

This would break rust-analyzer on configured-out crates, breaking eg project wide refactors.

Are you sure that this actually works? For example, say you have crates A and B and crate B uses crate A only if feature x is configured in. If you now do a refactor in crate A but feature x is configured out, will rust-analyzer change the source code of crate B?

We already have a make command for clippy.

Such a command does not integrate with IDEs. I.e. I want the IDE to show inline in the code editor where warnings occurred.

[bjorn3]

Are you sure that this actually works? For example, say you have crates A and B and crate B uses crate A only if feature x is configured in. If you now do a refactor in crate A but feature x is configured out, will rust-analyzer change the source code of crate B?

Yes, rust-analyzer works across all crates in the project even those crates that are unused. It does not however work on individual modules or functions that configured out.

Such a command does not integrate with IDEs. I.e. I want the IDE to show inline in the code editor where warnings occurred.

Rust-analyzer supports overriding the check command for as long as it supports --message-format json. Does IntelliJ also support overriding the check command? We can add an option to pass --error-format json to rustc. (--message-format json causes cargo to pass --error-format json to rustc and adds a bunch of extra cargo json messages that rust-analyzer only uses for progress reporting)

[mahkoh]

It would be easy to include these flags via .cargo/config.toml in the generated files but I see no way to figure out what those flags are from the rust-project.toml. I guess the user would have to manually maintain them?

I've looked at this and it seems that only some crates have no_std passed in via such a command line argument since proc macros actually do depend on std. So blindly passing -Zcrate-attr=no_std will not work. How does rust-analyzer know that std is not available in some crates but available in others?

Having such crates declare #![no_std] at the top seems much simpler and not much to ask. And several crates already do this.

Yes, rust-analyzer works across all crates in the project even those crates that are unused. It does not however work on individual modules or functions that configured out.

So, in practice, you have to configure everything in anyway if you want to do a reliable tree-wide refactor, since otherwise you can never know if the thing you're refactoring is used in some configured-out module or function.

Rust-analyzer supports overriding the check command for as long as it supports --message-format json. Does IntelliJ also support overriding the check command?

Afaict it does not.

[bjorn3]

How does rust-analyzer know that std is not available in some crates but available in others?

Rust-analyzer mostly uses #![no_std] to prefer importing from libcore over libstd and for it's cargo check integration. It doesn't really care otherwise if a crate is #![no_std] or not. For preferring importing libcore over libstd, I think the fact that the rust-project.json file doesn't mention libstd as dependency is enough to have the same effect.

Rust-analyzer supports overriding the check command for as long as it supports --message-format json. Does IntelliJ also support overriding the check command?

Afaict it does not.

That seems like a big omission.

[mahkoh]

I think the fact that the rust-project.json file doesn't mention libstd as dependency is enough to have the same effect.

I don't think that is the case since rust-project.json does not have to explicitly mention any sysroot crates. E.g. the rust-project.json generated by meson does not even if the code uses std, core, etc.

[bjorn3]

There are two ways to handle the sysroot in rust-project.json. One way is to set the sysroot source path (sysroot_src) in which case rust-analyzer will automatically add sysroot dependencies to every crate. This is what Meson does: https://github.com/mesonbuild/meson/blob/a7ea0cd7e4bc0d89d607d5fc0e999a4a6ab5b531/mesonbuild/backend/ninjabackend.py#L725 RfL however doesn't set the sysroot source path so no implicit sysroot dependencies will be added: https://github.com/torvalds/linux/blob/e48e99b6edf41c69c5528aa7ffb2daf3c59ee105/scripts/generate_rust_analyzer.py#L189-L192 Instead it explicitly adds a dependency on core and not std: https://github.com/torvalds/linux/blob/e48e99b6edf41c69c5528aa7ffb2daf3c59ee105/scripts/generate_rust_analyzer.py#L164