Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Module declared inside macro is not checked #3253

Open
mcheshkov opened this issue Dec 16, 2018 · 11 comments · Fixed by #3600
Open

Module declared inside macro is not checked #3253

mcheshkov opened this issue Dec 16, 2018 · 11 comments · Fixed by #3600
Labels
a-macros a-mods Module resolution. bug Panic, non-idempotency, invalid code, etc.
Milestone

Comments

@mcheshkov
Copy link

When I declare modules like this

#[cfg(unix)]
pub mod stub_hal;

rustfmt will check module, irrespective of condition.

However, when I declare them using cfg-if crate rustfmt doesn't check module.

cfg_if! {
    if #[cfg(unix)] {
        pub mod stub_hal;
    }
}

Is is designed behavior? Or is macro parsing not implemented for now?

That's rustfmt 1.0.1-nightly (be13559 2018-12-10)

@topecongiro
Copy link
Contributor

Parsing macro is possible only when the macro has valid syntax (e.g. println!). Most macros, including cfg_if!, contains invalid syntax in it (e.g. if #[cfg(unix)]), so rustfmt just ignores it.

@topecongiro topecongiro added the bug Panic, non-idempotency, invalid code, etc. label Feb 12, 2019
@gnzlbg
Copy link
Contributor

gnzlbg commented Feb 12, 2019

FYI this means that we can't use rustfmt effectively in rust-lang-nursery/{stdsimd, packed_simd} nor in rust-lang/libc. This appears to be hard to solve.

@topecongiro topecongiro added this to the 1.3.0 milestone Apr 22, 2019
Centril added a commit to Centril/rust that referenced this issue May 21, 2019
…base-dir, r=michaelwoerister

Add stream_to_parser_with_base_dir

This PR adds `stream_to_parser_with_base_dir`, which creates a parser from a token stream and a base directory.

Context: I would like to parse `cfg_if!` macro and get a list of modules defined inside it from rustfmt so that rustfmt can format those modules (cc rust-lang/rustfmt#3253). To do so, I need to create a parser from `TokenStream` and set the directory of `Parser` to the same directory as the parent directory of a file which contains `cfg_if!` invocation. AFAIK there is no way to achieve this, and hence this PR.

Alternatively, I could change the visibility of `Parser.directory` from `crate` to `pub` so that the value can be modified after initializing a parser. I don't have a preference over either approach (or others, as long as it works).
Centril added a commit to Centril/rust that referenced this issue May 21, 2019
…base-dir, r=michaelwoerister

Add stream_to_parser_with_base_dir

This PR adds `stream_to_parser_with_base_dir`, which creates a parser from a token stream and a base directory.

Context: I would like to parse `cfg_if!` macro and get a list of modules defined inside it from rustfmt so that rustfmt can format those modules (cc rust-lang/rustfmt#3253). To do so, I need to create a parser from `TokenStream` and set the directory of `Parser` to the same directory as the parent directory of a file which contains `cfg_if!` invocation. AFAIK there is no way to achieve this, and hence this PR.

Alternatively, I could change the visibility of `Parser.directory` from `crate` to `pub` so that the value can be modified after initializing a parser. I don't have a preference over either approach (or others, as long as it works).
Centril added a commit to Centril/rust that referenced this issue May 22, 2019
…base-dir, r=michaelwoerister

Add stream_to_parser_with_base_dir

This PR adds `stream_to_parser_with_base_dir`, which creates a parser from a token stream and a base directory.

Context: I would like to parse `cfg_if!` macro and get a list of modules defined inside it from rustfmt so that rustfmt can format those modules (cc rust-lang/rustfmt#3253). To do so, I need to create a parser from `TokenStream` and set the directory of `Parser` to the same directory as the parent directory of a file which contains `cfg_if!` invocation. AFAIK there is no way to achieve this, and hence this PR.

Alternatively, I could change the visibility of `Parser.directory` from `crate` to `pub` so that the value can be modified after initializing a parser. I don't have a preference over either approach (or others, as long as it works).
Centril added a commit to Centril/rust that referenced this issue May 22, 2019
…base-dir, r=michaelwoerister

Add stream_to_parser_with_base_dir

This PR adds `stream_to_parser_with_base_dir`, which creates a parser from a token stream and a base directory.

Context: I would like to parse `cfg_if!` macro and get a list of modules defined inside it from rustfmt so that rustfmt can format those modules (cc rust-lang/rustfmt#3253). To do so, I need to create a parser from `TokenStream` and set the directory of `Parser` to the same directory as the parent directory of a file which contains `cfg_if!` invocation. AFAIK there is no way to achieve this, and hence this PR.

Alternatively, I could change the visibility of `Parser.directory` from `crate` to `pub` so that the value can be modified after initializing a parser. I don't have a preference over either approach (or others, as long as it works).
@topecongiro
Copy link
Contributor

This seems to be not fixed in 1.3.0 😞

@topecongiro topecongiro reopened this Jun 25, 2019
@topecongiro topecongiro modified the milestones: 1.3.0, 1.4.0 Jul 15, 2019
@calebcartwright
Copy link
Member

calebcartwright commented Aug 29, 2019

@topecongiro - I believe I've got a fix for the cfg_if support (at least it's working my local env 😄)

Will open a PR shortly for review

@SOF3
Copy link

SOF3 commented Nov 7, 2019

This issue also exists with dirmod users. Modules are entirely generated by the macro, so it is impossible for rustfmt to detect them with heuristics like cfg_if! did.

A useful alternative would be to allow a flag like -r that formats all files in a directory recursively, no matter explicitly included or not, similar to the suggestion in #2426.

@dylni
Copy link
Contributor

dylni commented Jul 29, 2020

Parsing macro is possible only when the macro has valid syntax (e.g. println!).

@topecongiro Does this work for modules? This module in os_str_bytes isn't formatted, but the macro uses valid syntax:

https://github.com/dylni/os_str_bytes/blob/96b3151bec06d0309f2676e835fb1ed065cb9a53/src/lib.rs#L165-L167

@SimonSapin
Copy link

I’ve run into this bug.

Parsing macro is possible only when the macro has valid syntax (e.g. println!). Most macros, including cfg_if!, contains invalid syntax in it (e.g. if #[cfg(unix)]), so rustfmt just ignores it.

That explains why rustfmt cannot easily format the tokens in a macro invocation. However, couldn’t it expand macros when looking for mod items in order to find which other files to format?

@SOF3
Copy link

SOF3 commented Apr 12, 2021

That'd be too costly for a formatter that's supposed to only process the lexical level of the crate. In particular, procedural macros require compilation. If the procedural macro fails to compile, that would result in an error, which breaks backward compatibility of rustfmt.

@SimonSapin
Copy link

Right, drawing the line at proc macros makes sense to me. Would same-crate macro_rules also be too costly?

@daniel5151
Copy link

Just ran into this bug myself. gdbstub uses a declarative macro to define modules for different protocol packets, and I recently noticed that the CI wasn't enforcing formatting in them.

Is there any suggested workaround / fix on the horizon? It seems I might need to update the CI to manually invoke rustfmt on each file manually...

@Purpzie
Copy link

Purpzie commented Oct 20, 2021

Currently running into this problem as well.

While it does make sense not to parse every macro like that, what if there was a #[rustfmt::follow_modules] attribute or something similar? It would go on macros where you want rustfmt to parse and find modules.

#[rustfmt::follow_modules]
macro_rules! custom_macro {
    ($(mod $mod_name:ident;)*) => {
        // do something here...
    }
}

custom_macro! {
    mod foo;
    mod bar;
    mod baz;
}

@ytmimi ytmimi added the a-mods Module resolution. label Dec 21, 2021
Luminiscental added a commit to Luminiscental/aoc2021 that referenced this issue Dec 21, 2021
cargo-fmt skips files because of rust-lang/rustfmt#3253
bors bot added a commit to nix-rust/nix that referenced this issue Nov 12, 2022
1854: Reformat everything r=rtzoeller a=SUPERCILEX

Unfortunately, #1748 didn't do the trick because of rust-lang/rustfmt#3253. This PR fully enforces global formatting.

Closes #770

Co-authored-by: Alex Saveau <saveau.alexandre@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a-macros a-mods Module resolution. bug Panic, non-idempotency, invalid code, etc.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants