[Tracking issue] New lint metadata collection (a metadata collection monster) #7172

xFrednet · 2021-05-05T17:52:45Z

This issue tracks the effort to replace the current lint documentation collection with a new Rust implementation. The implementation will be based on a lint to access rustc's type information (See #4310 for further information). The first incomplete version of the metadata collection monster (as I like to call it) got merged in #6887.

To-Do list

Needs investigation / discussion

Look into speeding up the collection by only registering the collection lint itself. (Suggested by @flip1995 here) (Related discussion about registration)
- Done by flip1995 in PR Early return from LintPass registration when collecting metadata #7253
Maybe include the lint output for the examples if it's not too much effort. (Suggested here)

@rustbot label +C-tracking-issue

The text was updated successfully, but these errors were encountered:

…lip1995 Metadata collection monster eating deprecated lints This adds the collection of deprecated lints to the metadata collection monster. The JSON output has the same structure with the *new* lint group "DEPRECATED". Here is one of fourteen examples it was able to dig up in Clippy's code: ```JSON { "id": "assign_op_pattern", "id_span": { "path": "src/assign_ops.rs", "line": 34 }, "group": "clippy::style", "docs": " **What it does:** Checks for `a = a op b` or `a = b commutative_op a` patterns.\n\n **Why is this bad?** These can be written as the shorter `a op= b`.\n\n **Known problems:** While forbidden by the spec, `OpAssign` traits may have\n implementations that differ from the regular `Op` impl.\n\n **Example:**\n ```rust\n let mut a = 5;\n let b = 0;\n // ...\n // Bad\n a = a + b;\n\n // Good\n a += b;\n ```\n", "applicability": { "is_multi_part_suggestion": false, "applicability": "MachineApplicable" } } ``` And you! Yes you! Sir or Madam can get all of this **for free** in Clippy if this PR gets merged. (Sorry for the silliness ^^) --- See: #7172 for the full metadata collection to-do list or to suggest a new feature in connection to it 🙃 --- changelog: none r? `@flip1995`

…r=flip1995 Add `cargo collect-metadata` as an cargo alias for the metadata collection lint This PR adds a new alias to run the metadata collection monster on `clippy_lints`. I'm currently using it to create the `metadata_collection.json` file and I plan to use it in the `deply.sh` script. Having it as a new alias enables us to simply use: ```sh cargo collect-metadata ``` It sometimes requires running `cargo clean` before collecting the metadata due to caching. I'm still debating if I should include a cargo clean as part of the `run_metadata_collection_lint` test or not. Input on this would be greatly appreciated 🙃 That's it, just a small change that can be reviewed and merged in parallel to #7214. --- See: #7172 for the full metadata collection to-do list or to suggest a new feature in connection to it. --- changelog: none r? `@flip1995` btw. feel free to pass these PRs one to other team members as well if you want.

…995,camsteffen Metadata collection monster searching for Clippy's configuration options This PR teaches our lovely metadata collection monster which configurations are available inside Clippy. It then adds a new *Configuration* section to the lint documentation. --- The implementation uses the `define_Conf!` macro to create a vector of metadata during compilation. This enables easy collection and parsing without the need of searching for the struct during a lint-pass (and it's quite elegant IMO). The information is then parsed into an intermediate struct called `ClippyConfiguration` which will be saved inside the `MetadataCollector` struct itself. It is currently only used to generate the *Configuration* section in the lint documentation, but I'm thinking about adding an overview of available configurations to the website. Saving them in this intermediate state without formatting them right away enables this in the future. The new parsing will also allow us to have a documentation that spans over multiple lines in the future. For example, this will be valid when the old script has been removed: ```rust /// Lint: BLACKLISTED_NAME. /// The list of blacklisted names to lint about. NB: `bar` is not here since it has legitimate uses (blacklisted_names: Vec<String> = ["foo", "baz", "quux"].iter().map(ToString::to_string).collect()) ``` The deprecation reason is also currently being collected but not used any further and that's basically it. --- See: #7172 for the full metadata collection to-do list or to suggest a new feature in connection to it 🙃 --- changelog: none r? `@flip1995` cc `@camsteffen` It would be great if you could also review this PR as you have recently worked on Clippy's `define_Conf!` macro.

Adding the default lint level to the metadata collection I noticed while working on the website adaption that the lint groups still had the `clippy::` prefix in the JSON output. This PR removes this prefix and adds a `level` field to each lint and with that simplifies the website display and saves performance. The deprecated lints get are assigned to the level `none`. This is a bit different in comparison to the current lint list, but I believe that this will look better overall. Unless there is any argument against this :). That's it just a small baby PR in comparison to the original monster ^^ --- See: #7172 for the full metadata collection to-do list or to suggest a new feature in connection to it. --- changelog: none r? `@flip1995`

flip1995 · 2021-05-20T10:10:08Z

Found a "bug" while syncing: If you re-run the cargo collect-metadata command, it won't re-produce the json file again, since it thinks that it already ran on the crate and no re-check is necessary.

xFrednet · 2021-05-20T10:40:32Z

Jup, this was also mentioned in #7217. Is there a simple way to force a re-check. I usually used cargo clean which seems a bit excessive though?

flip1995 · 2021-05-20T11:04:46Z

There's a hacky way in touching clippy_lints/src/lib.rs. So touch clippy_lints/src/lib.rs && cargo collect-metadata will re-run the tool, but will also re-build clippy before re-running the tool. cargo has no --force flag though (AFAIK).

xFrednet · 2021-05-20T11:17:37Z

Using touch is still quicker than cleaning the entire project. We could maybe include a check in the run_metadata_collection_lint() test that touches the file and reruns the collection if the JSON output file was not updates or doesn't exist. It's hacky but a bit better than touching the file each time. What do you think?

flip1995 · 2021-05-20T11:34:57Z

Yes, that was also my idea.

xFrednet · 2021-05-20T11:50:26Z

Okay, would you like to do this change as part of the CI, or shall I have a look into it? 🙃

flip1995 · 2021-05-20T12:01:18Z

Okay, would you like to do this change as part of the CI, or shall I have a look into it?

It doesn't matter for CI, since Clippy will be rebuild for this anyway. :D

xFrednet · 2021-05-20T12:09:14Z

Okay, guess it's an item on my to-do list then ^^. The index.html adaption should be completed soon as well.

Early return from LintPass registration when collecting metadata This speeds up the metadata collection by 2-2.5x on my machine. During metadata collection other lint passes don't have to be registered, only the lints themselves. cc #7172 r? `@xFrednet` changelog: none

xFrednet · 2021-05-20T14:19:36Z

I've adapted the website to use the new lints.json format in xfrednet:7172-adapt-website-to-new-format. A preview of the changes can be seen here: https://xfrednet.github.io/rust-clippy/master/index.html.

I'm not particularly good when it comes to designing GUIs. Any feedback is welcome 🙃.

…on, r=flip1995 Adding the ability to invalidate caches to force metadata collection This adds the discussed hack to touch `clippy_lints/src/lib.rs` to invalidate cargos cache and force metadata collection. I've decided to use the [`filetime`](https://github.com/alexcrichton/filetime) crate instead of the touch command to make is cross-platform and just cleaner in general. Looking at the maintainers I would say that it's a save crate to use xD. --- cc: #7172 I know this ID without looking it up now... This is not good changelog: none r? `@flip1995`

Fixed broken deploy script due to multiline configuration docs The deploy script on master currently runs into an error (See [log](https://github.com/rust-lang/rust-clippy/runs/2865828873)) due to the new configuration documentation added in #7299. The current documentation collection for the configuration macro sadly doesn't support multiline doc comments. This will be changes in the future with the new metadata collector tracked in #7172 For now we have to use `<br>` inside doc comments to add paragraphs. This PR restricts `define_Conf!` macro to single lines and adds a comment explaining the reasoning behind it. It also adjusted the actual document parsing to fix a bug. (The parsing was automatically stopping on the first curly bracket, even if it was part of a doc comment). changelog: none

…ormat, r=flip1995 Adapting the lint list to Clippy's new metadata format This is close to the end of a long living project to rewrite the lint metadata collection for Clippy. Progress on this has been tracked in rust-lang#7172. This PR adds one of the last missing puzzle pieces, the actual display of all the changes that have been done in the background. A preview can be seen here: [Clippy's lint list](https://xfrednet.github.io/rust-clippy/master/index.html) The styling has been discussed on [zulip](https://rust-lang.zulipchat.com/#narrow/stream/257328-clippy/topic/Styling.20feedback.20for.20Clippy's.20lint.20list/near/239601067) but is still open to suggestion. Side note: It has been fun working on the website where we don't have unit tests and everything is just tested by playing around. However, it's good that this chaos is contained into this one part of Clippy. 🐛 --- Closes: rust-lang#1303 Closes: rust-lang#4310 This actually closes fewer things than I thought it would... changelog: Reworked Clippy's [website](https://rust-lang.github.io/rust-clippy/master/index.html): changelog: * Added applicability information about lints changelog: * Added a link to jump to the specific lint implementation changelog: * Adapted some styling and improved loading time r? `@flip1995`

…ion, r=flip1995 Update lint documentation to use markdown headlines This PR updates all lint documentation to use markdown headlines. It additionally removed the *Known problems* section for lints without any problems. I've double-checked all automatic replacements, but a second pair of eyes is definitely appreciated! I wasn't sure when you wanted to switch to the new metadata collection tomorrow, I therefore prepared this PR today. And that's it this is a standalone PR to keep the other related PRs reviewable. changelog: none r? `@flip1995` cc: rust-lang#7172 Note: This should be merged with the other metadata collection related PRs.

…ormat, r=flip1995 Adapting the lint list to Clippy's new metadata format This is close to the end of a long living project to rewrite the lint metadata collection for Clippy. Progress on this has been tracked in rust-lang#7172. This PR adds one of the last missing puzzle pieces, the actual display of all the changes that have been done in the background. A preview can be seen here: [Clippy's lint list](https://xfrednet.github.io/rust-clippy/master/index.html) The styling has been discussed on [zulip](https://rust-lang.zulipchat.com/#narrow/stream/257328-clippy/topic/Styling.20feedback.20for.20Clippy's.20lint.20list/near/239601067) but is still open to suggestion. Side note: It has been fun working on the website where we don't have unit tests and everything is just tested by playing around. However, it's good that this chaos is contained into this one part of Clippy. 🐛 --- Closes: rust-lang#1303 Closes: rust-lang#4310 This actually closes fewer things than I thought it would... changelog: Reworked Clippy's [website](https://rust-lang.github.io/rust-clippy/master/index.html): changelog: * Added applicability information about lints changelog: * Added a link to jump to the specific lint implementation changelog: * Adapted some styling and improved loading time r? `@flip1995`

…ion, r=flip1995 Update lint documentation to use markdown headlines This PR updates all lint documentation to use markdown headlines. It additionally removed the *Known problems* section for lints without any problems. I've double-checked all automatic replacements, but a second pair of eyes is definitely appreciated! I wasn't sure when you wanted to switch to the new metadata collection tomorrow, I therefore prepared this PR today. And that's it this is a standalone PR to keep the other related PRs reviewable. changelog: none r? `@flip1995` cc: rust-lang#7172 Note: This should be merged with the other metadata collection related PRs.

Add Clippy version to Clippy's lint list Hey, hey, the semester is finally over, and I wanted to get back into hacking on Clippy. It has also been some time since our metadata collection monster has been feed. So, this PR adds a new attribute `clippy::version` to document which version a lint was stabilized. I considered using `git blame` but that would be very hacky and probably not accurate. I'm also thinking that this attribute can be used to have a `clippy::nightly` lint group which is allow-by-default that delays setting the actual lint group until the defined version is reached. Just something to consider regarding #6623 🙃 This PR only adds the version to 4 lints to keep it reviewable. I'll do a followup PR to add the version to other lints if the implementation is accepted 🙃 ![image](https://user-images.githubusercontent.com/17087237/137118859-0aafdfdf-7595-4289-8ba4-33d58eb6991d.png) Also, mobile approved xD ![image](https://user-images.githubusercontent.com/17087237/137118944-833cf7fb-a4a1-45d6-9af8-32c951822360.png) --- r? `@flip1995` cc: #7172 closes: #6492 changelog: [Clippy's lint list](https://rust-lang.github.io/rust-clippy/master/index.html) now displays the version a lint was added. 🎉 --- Example lint declaration after this update: ```rs declare_clippy_lint! { /// [...] /// /// ### Example /// ```rust /// // Bad /// let x = 3.14; /// // Good /// let x = std::f32::consts::PI; /// ``` #[clippy::version = "pre 1.29.0"] pub APPROX_CONSTANT, correctness, "the approximate of a known float constant (in `std::fXX::consts`)" } ```

Collect renamed lints changelog: none cc #7172 r? `@xFrednet`

Collect renamed lints changelog: Display past names of renamed lints on Clippy's lint list cc #7172 r? `@xFrednet`

Add lint output to lint list changelog: Add the ability to show the lint output in the lint list This just adds the logic to produce the output, it hasn't been added to any lints yet. It did help find some mistakes in some docs though 😄. ### Screenshots <details> <summary>A single code block</summary> ![single-code-block](https://user-images.githubusercontent.com/69764315/172013766-145b22b1-1d91-4fb8-9cd0-b967a52d6330.png) </details> <details> <summary>A single code block with a "Use instead" section</summary> ![with-usage](https://user-images.githubusercontent.com/69764315/172013792-d2dd6c9c-defa-41e0-8c27-8e8e311adb63.png) </details> <details> <summary>Multiple code blocks</summary> ![multi-code-block](https://user-images.githubusercontent.com/69764315/172013808-5328f59b-e7c5-4914-a396-253822a6d350.png) </details> This is the last task in #7172 🎉. r? `@xFrednet` (?)

xFrednet · 2022-10-12T18:55:53Z

Looking at the lint description, it looks like it's finally time to close this issue for good. @flip1995 Thank you for the mentoring on this one. This really introduced me to Clippy, and it's nice to see that the infrastructure still runs <3

rustbot added the C-tracking-issue Category: Tracking Issue label May 5, 2021

flip1995 added A-infra Area: CI issues and issues that require full access for GitHub/CI C-an-interesting-project Category: Interesting projects, that usually are more involved design/code wise. A-website Area: Improving the clippy website labels May 5, 2021

flip1995 mentioned this issue May 6, 2021

Conf macro improvements part 2 #7154

Merged

xFrednet mentioned this issue May 8, 2021

Metadata collection monster eating deprecated lints #7197

Merged

This was referenced May 12, 2021

Metadata collection monster searching for Clippy's configuration options #7214

Merged

Add cargo collect-metadata as an cargo alias for the metadata collection lint #7217

Merged

This was referenced May 16, 2021

Adding a dev website path for the new lints.json format and testing deployment #7229

Closed

Adding the default lint level to the metadata collection #7246

Merged

flip1995 mentioned this issue May 20, 2021

Early return from LintPass registration when collecting metadata #7253

Merged

xFrednet mentioned this issue May 20, 2021

Adding the ability to invalidate caches to force metadata collection #7256

Merged

xFrednet mentioned this issue May 26, 2021

Adapting the lint list to Clippy's new metadata format #7279

Merged

xFrednet mentioned this issue Jun 20, 2021

Fixed broken deploy script due to multiline configuration docs #7385

Merged

xFrednet mentioned this issue Jul 2, 2021

Update lint documentation to use markdown headlines #7420

Merged

xFrednet mentioned this issue Oct 12, 2021

Add Clippy version to Clippy's lint list #7813

Merged

This was referenced Nov 11, 2021

Improve filtering options for Clippy's lint list #7958

Closed

Add copy lint name function to Clippy's lint list #7959

Closed

xFrednet mentioned this issue May 18, 2022

Add copy lint name button #8839

Merged

Serial-ATA mentioned this issue May 19, 2022

Collect renamed lints #8843

Merged

bors added a commit that referenced this issue Jun 4, 2022

Auto merge of #8843 - Serial-ATA:collect-renamed, r=xFrednet

c7893da

Collect renamed lints changelog: none cc #7172 r? `@xFrednet`

bors added a commit that referenced this issue Jun 4, 2022

Auto merge of #8843 - Serial-ATA:collect-renamed, r=xFrednet

8ef4908

Collect renamed lints changelog: Display past names of renamed lints on Clippy's lint list cc #7172 r? `@xFrednet`

Serial-ATA mentioned this issue Jun 4, 2022

Add lint output to lint list #8947

Merged

xFrednet closed this as completed Oct 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Tracking issue] New lint metadata collection (a metadata collection monster) #7172

[Tracking issue] New lint metadata collection (a metadata collection monster) #7172

xFrednet commented May 5, 2021 •

edited

Loading

flip1995 commented May 20, 2021

xFrednet commented May 20, 2021

flip1995 commented May 20, 2021

xFrednet commented May 20, 2021

flip1995 commented May 20, 2021

xFrednet commented May 20, 2021

flip1995 commented May 20, 2021

xFrednet commented May 20, 2021

xFrednet commented May 20, 2021

xFrednet commented Oct 12, 2022

[Tracking issue] New lint metadata collection (a metadata collection monster) #7172

[Tracking issue] New lint metadata collection (a metadata collection monster) #7172

Comments

xFrednet commented May 5, 2021 • edited Loading

To-Do list

Needs investigation / discussion

flip1995 commented May 20, 2021

xFrednet commented May 20, 2021

flip1995 commented May 20, 2021

xFrednet commented May 20, 2021

flip1995 commented May 20, 2021

xFrednet commented May 20, 2021

flip1995 commented May 20, 2021

xFrednet commented May 20, 2021

xFrednet commented May 20, 2021

xFrednet commented Oct 12, 2022

xFrednet commented May 5, 2021 •

edited

Loading