Tracking issue: RFC 2103 - attributes for tools

[nrc]

Support scoped attributes for white-listed tools, e.g., #[rustfmt::skip]

RFC
discussion thread

• implement for attributes (mentoring instructions here)
• implement for lints (#52018; see #44690 (comment))
• Adjust documentation (see instructions on forge)
• Stabilization PR (see instructions on forge)

Current status: stabilised for attributes, some work still to do on lints.

[rep-nop]

As mentioned on the Gitter chat, I'm willing to take a try at this! Should just need pointed in the right direction :)

[nikomatsakis]

@petrochenkov I suspect you'd be the best one to write up some mentoring instructions here. Any chance you can put some thought into it?

[rep-nop]

I'll definitely be getting myself familiar with the parser and AST in the mean time, so no big rush!

[nikomatsakis]

OK, since @petrochenkov has been busy, let me take a shot at some notes. Perhaps he can correct me if he thinks things could be done a better way.

To start, the structure that represents attributes in the AST is called Attribute. From what I can tell, the parser already parses attributes and permits them to contain general paths.

Basically, the model is this: within the AST, an attribute is just a token tree. However, sometimes, the compiler tries to parse and impose structure on this tree by parsing it as something like #[derive(Foo)]. This parsed form is represented as a MetaItem struct.

This parsing that converts an Attribute into a MetaItem is when you wind up with an error due to something like #[foo::bar(...)]. In particular, such a path is flagged as an error by the helper parse_meta. One part of this RFC then is just to tweak this check to permit some cases of non-identifier paths.

However, we must also decide how to expand MetaItem -- right now, it's name field has type Name -- i.e., a single name -- but we now want to represent compound names like a::b. The hackiest thing to do would be to intern the combined string as a big string. A less hacky thing would probably be to change Name into Vec<Name> -- or perhaps an enum that chooses between a single name and multiple names. I'm not sure which is the best way forward there.

[nikomatsakis]

@rep-nop hey, I'm just checking in -- are you still hacking on this? How's it going?

[rep-nop]

@nikomatsakis sorry, school has been keeping me fairly busy lately but things look like they're starting to lighten up for now! I spent ~2ish hours about 2 weeks ago on it and was getting somewhere I think, but I'm going to be hopefully trying a new approach that I hope will work better within the week (if not today even!)

[nikomatsakis]

@rep-nop great =) Let me know if I can be of any assistance.

[rep-nop]

Okay so while doing a more thorough read through the RFC I came up with a quick requirements document to track progress on it (below, it wont let me attach it for some reason? Let me know if I missed anything major or important!). The only part of the RFC I don't think I understand very well is the "Activation and unused attributes/lints" section. It talks about rustdoc, for example, not being "activated" until needed, what exactly does that imply?

RFC 2103 - Attributes for Tools Requirements

┌───────────┐
│ Long-term │
└───────────┘
[x] Some unspecified opt-in mechanism
    │
    ├> #[cfg_tool(mytool)] ?
    │ 
    └> #[allow_tool(mytool)] ?

┌──────────────────┐
│ RFC Requirements │
└──────────────────┘
[x] Allow any tool that ships with Rust to be included
    │   by default
    │
    └> { rustc, rustdoc, rls } (should I add clippy and rustfmt? the RFC talks about adding them "soon")

[x] Attributes must be long-lived in compilation

[x] Compiler doesn't warn on unused/unknown attributes,
    │   however tools must
    │
    │ Allowed by compiler
    ├> #[rustfmt::foo] 
    │ Tool produced warning/error
    └> ex. "rustfmt: warn: `foo` is an unknown attribute"

[rep-nop]

@nikomatsakis @nrc any input? :)

[nikomatsakis]

@rep-nop seems reasonable. =)

Activation and unused attributes/lints

I think what this means it that we won't accept rustdoc:foo attributes until rustdoc starts having a use for them.

[rep-nop]

@nikomatsakis so is that something that we're going to have to keep track of or..? Also, any idea where the whitelist will be located inside of the compiler? Currently, I just have it inside of the function as an array and the function checks against just the tool name (so should I make an attribute list for each one of them also?) and error if its not one of the whitelisted tools. The RFC talks about adding tools to that list from inside the source, however it looks like it wasn't decided on what that'd look like. @petrochenkov any suggestions/tips for a proper implementation?

[petrochenkov]

@rep-nop
I don't know how the proper implementation should look exactly.
Name resolution for attributes is kinda mess because it works with all kinds of attributes - built in, custom, derives, legacy derives, procedural macros, legacy procedural macros. I never looked at that code carefully.
Someone needs to figure out how it works exactly and how attributes for tools fit into it.
(I can figure it out myself, but then it would be much faster and easier to proceed and implement this myself as well, mentoring is harder and more time consuming.)

[nrc]

so is that something that we're going to have to keep track of or..?

Yes. There are essentially two lists - one of lints and one of attributes. The compiler only cares about the active tools (initially both lists will be empty). So if the active attributes whitelist is rustfmt, clippy and the active lints whitelist is clippy, then a rustfmt:: lint would still be an error (as would an rls:: attribute).

should I add clippy and rustfmt? the RFC talks about adding them "soon"

Yes, they are both on their way.

The requirements looks good.

Also, any idea where the whitelist will be located inside of the compiler

Good question! The lists could be provided by the build system, probably as an env var (or two). It should live somewhere that deals with tools. They could be hard-wired into the compiler, but I'm not really sure where I would put it. I don't think we need to worry about adding tools to the whitelist, we can do that manually.

In terms of implementation, I think the checking of the whitelist names in attributes should happen either during macro name resolution or just afterwards. The key to when to do the check is about the behaviour we want with precedence (tool vs macro). If you have questions, @jseyfried is probably the best person to ask.

[rep-nop]

Thanks for the answers! :) @nrc right now I have it checking the names at name resolution when it detects a path inside the attribute, which wasn't too difficult to get a quick and crude implementation done (which I believe is linked above in a commit I did). I'll look into using an env var to specify tool names!

[topecongiro]

@rep-nop Do you have any update? This is (potentially) blocking rustfmt from reaching 1.0, so I would like to push this forward. If you do not have enough time to work on this, I am willing to take a shot at this.

[rep-nop]

@topecongiro ah if this is going to be blocking rustfmt I'll release it as I'm going to be fairly busy this upcoming semester. I was unfortunately not able to progress as much as I'd liked to have for various reasons, so have at it! :)

[flip1995]

I would like to continue working on this, except @topecongiro wants to pick this up again?

Since #47773 already implemented the tool_attributes part, the next step would be doing the same for tool_lints?

[topecongiro]

@flip1995 I do not think that I can update the initial PR any time soon. So it will be great if you are going to work on this!

[nrc]

This is only partially stable.

[aturon]

@nrc Is this stable "enough" for the edition? Can it be removed from the milestone?

[nrc]

@nrc Is this stable "enough" for the edition? Can it be removed from the milestone?

Yes!

[sanmai-NL]

@nrc: kind reminder to update the opening post with the current progress.

[Nemo157]

This appears to be stablilized, should the OP checkbox be ticked?