(Big performance change) Do not run lints that cannot emit #125116

blyxyas · 2024-05-14T11:20:44Z

Before this lint, adding a lint was a difficult matter because it always had some overhead involved. This was because all lints would run, no matter their default level, or if the user had #![allow]ed them. This PR changes that. This change would improve both the Rust lint infrastructure and Clippy, but Clippy will see the most benefit, as it has about 900 registered lints (and growing!)

So yeah, with this little patch we filter all lints pre-linting, and remove any lint that is either:

Manually #![allow]ed in the whole crate,
Allowed in the command line, or
Not manually enabled with #[warn] or similar, and its default level is Allow

As some lints need to run, this PR also adds loadbearing lints. On a lint declaration, you can use the [loadbearing: true] marker to label it as loadbearing. A loadbearing lint will never be filtered.

Phase 1/2 Not all lints are being filtered, I'm still working on it, but this branch still gives us about a 2% improvement, so why not merge it already.

Fixes #106983

rustbot · 2024-05-14T11:20:52Z

r? @michaelwoerister

rustbot has assigned @michaelwoerister.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2024-05-14T11:20:54Z

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

Some changes occurred in src/tools/clippy

cc @rust-lang/clippy

blyxyas · 2024-05-14T11:22:05Z

cc @nnethercote @Kobzol, the perf wizards. Could you please give this PR a look and tell me if there are any obvious performance issues on the filtering?

matthiaskrgr · 2024-05-14T11:26:37Z

@bors try @rust-timer queue

…r=<try> (Big performance change) Do not run lints that cannot emit Before this lint, adding a lint was a difficult matter because it always had some overhead involved. This was because all lints would run, no matter their default level, or if the user had `#![allow]`ed them. This PR changes that. This change would improve both the Rust lint infrastructure and Clippy, but Clippy will see the most benefit, as it has about 900 registered lints (and growing!) So yeah, with this little patch we filter all lints pre-linting, and remove any lint that is either: - Manually `#![allow]`ed in the whole crate, - Allowed in the command line, or - Not manually enabled with `#[warn]` or similar, and its default level is `Allow` I open this PR to receive some feedback, mainly related to performance. We have lots of `Lock`s, `with_lock` and similar functions (also lots of cloning), so the filtering performance is not the best. In an older iteration, instead of doing this in the parsing phase, we developed a visitor with the same function but without so many locks, would reverting to that change help? I'm not sure tbh.

bors · 2024-05-14T11:27:49Z

⌛ Trying commit 7606f89 with merge cc1d40f...

Kobzol · 2024-05-14T11:54:10Z

@lqd haven't you tried something like this before? 🤔

bors · 2024-05-14T13:04:11Z

☀️ Try build successful - checks-actions
Build commit: cc1d40f (cc1d40f134ee8336cbb7c7561deaed4aa5906e0e)

lqd · 2024-05-14T13:14:38Z

@lqd haven't you tried something like this before? 🤔

We've tried a few different things yes, and so has blyxyas -- it maybe wasn't exactly like this, but I encountered annoyances like: some slow const eval loadbearing lint that shouldn't be ignored, lints that would be allowed unexpectedly because cargo allows lints unconditionally on dependencies (arguably the most common usage, and where perf gains would show up AFAICT) but some may trigger FCWs or are required to lint on dependencies despite being allowed, et cetera.

Refactoring and fixing all these were too costly compared to the gains at the time, as rustc's lints were fast enough on dependencies, also a "rarer" use-case. That being said, we've added and uplifted more lints since then, including possibly costly ones like the non local impls one, and the situation may also be different for clippy itself (but we won't see that in the perf.rlo results, only locally with the clippy dedicated commands IIUC)

rust-timer · 2024-05-14T14:45:34Z

Finished benchmarking commit (cc1d40f): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.5%]	25
Regressions ❌ (secondary)	0.4%	[0.1%, 1.6%]	11
Improvements ✅ (primary)	-0.4%	[-0.4%, -0.4%]	1
Improvements ✅ (secondary)	-0.1%	[-0.1%, -0.1%]	1
All ❌✅ (primary)	0.3%	[-0.4%, 0.5%]	26

Max RSS (memory usage)

Results (primary 2.3%, secondary -0.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.3%	[2.3%, 2.3%]	1
Regressions ❌ (secondary)	3.6%	[2.0%, 5.3%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.7%	[-5.2%, -4.1%]	2
All ❌✅ (primary)	2.3%	[2.3%, 2.3%]	1

Cycles

Results (secondary 2.4%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.4%	[1.5%, 3.1%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 676.788s -> 676.098s (-0.10%)
Artifact size: 316.11 MiB -> 316.15 MiB (0.01%)

Centri3 · 2024-05-14T14:54:05Z

The benchmark doesn't check clippy, right? As lqd hinted at as well? And without splitting allow-by-default rustc lints it does nothing without clippy, so I think this just shows how much time it takes to filter them (can someone else confirm this :3c)

Thus, basically nothing it seems :3 (So @blyxyas maybe the cloning is ok?)

blyxyas · 2024-05-15T08:51:10Z

The benchmark doesn't check clippy, right?

Yeah, the benchmarks currently doesn't check Clippy, that's why I'm currently benchmarking on a different server via SSH (A server that we got explicitly to benchmark Clippy). I'll post the results here when they arrive :)

Also, it currently doesn't check builtin lints because I'm having some issues checking that. That's also part of why I decided to open the PR, maybe someone has some idea (I'll see if I can read the previous attempts by lqd, maybe I can learn something from them)

EDIT: Seems like lqd hasn't pushed their attempts, I'll have to keep trying new approaches by myself.

blyxyas · 2024-05-15T11:04:14Z

Okis, here are the results (Wall time, Clippy)

Wall time

❌
[ +0.45%, +190.50%]
+9.60% 81 (36)
✅
[-73.80%, -0.44%]
-7.00% 77 (36)
❌,✅
[-73.80%, +190.50%]
+1.51% 158 (44)

Max RSS

❌
[ +0.40%, +2.80%]
+1.17% 21 (16)
✅
[ -5.15%, -0.44%]
-1.60% 39 (24)
❌,✅
[ -5.15%, +2.80%]
-0.63% 60 (31)

Instructions

❌
[ +0.42%, +0.62%]
+0.52% 6 (4)
✅
[ -1.82%, -0.32%]
-0.74% 13 (7)
❌,✅
[ -1.82%, +0.62%]
-0.34% 19 (11)

Cycles

[ +0.42%, +25.48%]
+4.33% 91 (38)
✅
[-14.52%, -0.40%]
-3.33% 59 (32)
❌,✅
[-14.52%, +25.48%]
+1.31% 150 (43)

blyxyas · 2024-05-15T11:06:43Z

Those wall times are proof that this optimization has a lot of potential, the main drawback is that the filtering / parsing code is not fast enough, so in some scenarios that I'm not really able to determine exactly what do they have in common, the optimization goes backwards.

But a ~70% in Wall time, that's great and we should look more into it.

Kobzol · 2024-05-15T11:09:50Z

I wouldn't draw too many conclusions from these results, they seem to be quite unstable (there is also a 190% walltime regression). Note that even for PRs that don't have large perf. impacts, we can see ~30% walltime swings even on the stable benchmarking server (https://perf.rust-lang.org/compare.html?start=9e7aff794539aa040362f4424eb29207449ffce0&end=44fa5fd39a1d2af41bd7f43bc246a5e4f6d94696&stat=wall-time&nonRelevant=true).

blyxyas · 2024-05-15T17:13:50Z

I've changed the system, we're back to using visitors (I've benchmarked this new commit, it should have 0 regressions and about -0.66% improvement)

@bors try @rust-timer queue

bors · 2024-05-15T17:15:02Z

⌛ Trying commit 828cd60 with merge 68a9e31...

blyxyas · 2024-05-18T16:26:47Z

CI is not failing! ❤️

Manishearth · 2024-05-19T13:33:06Z

Excited to finally see this happening!! I'm off in the wilderness and can't take a closer look for a couple more days, but nice work so far!

Xiretza

I don't really have enough expertise to comment on the actual feature, but I did find a couple (non-blocking) nits.

compiler/rustc_lint/src/late.rs

compiler/rustc_middle/src/query/mod.rs

compiler/rustc_parse/src/parser/attr.rs

compiler/rustc_lint/src/late.rs

michaelwoerister · 2024-05-21T08:12:01Z

Thanks for the PR, @blyxyas!

Excited to finally see this happening!! I'm off in the wilderness and can't take a closer look for a couple more days, but nice work so far!

@Manishearth, do you want to do a review before this is approved? I plan to take a closer look at this within the next couple of days, assuming the code has reached a steady state. I don't really know this part of the compiler well, so expert input is appreciated.

blyxyas · 2024-05-21T12:09:01Z

I forgot to say: While this PR is finished (in my eyes). The "ignoring allowed lints" project is not yet finished.
The performance improvement is not as big as I'd like (we're probably not filtering some relevant passes?), but I'm not sure when I'll be able to keep investigating, so any improvement is welcomed!

Manishearth · 2024-05-21T14:26:44Z

@michaelwoerister no, I didn't plan to!

michaelwoerister · 2024-05-27T09:40:59Z

compiler/rustc_lint_defs/src/lib.rs

+
+    /// `true` if this lint should not be filtered out under any circustamces
+    /// (e.g. the unknown_attributes lint)
+    pub loadbearing: bool,


Can we give this a more descriptive name, like always_run or run_unconditionally or never_filter?

michaelwoerister · 2024-05-27T09:46:14Z

compiler/rustc_lint/src/levels.rs

+struct LintLevelMinimum<'tcx> {
+    tcx: TyCtxt<'tcx>,
+    /// The actual list of detected lints.
+    lints_to_emit: Vec<String>,


Would lints_to_run be a more accurate name for the field?

michaelwoerister · 2024-05-27T09:54:13Z

Question: Why is the tool name being removed? Doesn't that potentially lead to ambiguities?

michaelwoerister · 2024-05-27T09:56:29Z

compiler/rustc_lint/src/levels.rs

+    tcx: TyCtxt<'tcx>,
+    /// The actual list of detected lints.
+    lints_to_emit: Vec<String>,
+    lints_allowed: Vec<String>,


Can you turn these two fields into FxIndexSet to speed up the contains lookups?

bors · 2024-05-27T11:08:16Z

☔ The latest upstream changes (presumably #125410) made this pull request unmergeable. Please resolve the merge conflicts.

Polish marramiau miau miau 🎉 meowmeowmeow Removing unused files Meow format Make lint level minimums parallel Fix author lint + stop using symbols Move from symbols to strings + fix things meow sync Remove crate-checking code Prepare for WIP PR opening m Turn back to visitor Add loadbearing lints + Move filtering code to lint_crate Move lint filter to check_mod Also handle lint groups + Should work

rustbot assigned michaelwoerister May 14, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels May 14, 2024

blyxyas marked this pull request as draft May 14, 2024 11:23

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 14, 2024

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels May 14, 2024

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 15, 2024

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 16, 2024

blyxyas force-pushed the ignore-allowed-lints-final branch from 15cf7cd to 18c8f85 Compare May 18, 2024 13:22

This comment has been minimized.

Sign in to view

blyxyas force-pushed the ignore-allowed-lints-final branch from 4d12f19 to 718243c Compare May 19, 2024 14:10

blyxyas marked this pull request as ready for review May 19, 2024 14:15

This comment has been minimized.

Sign in to view

blyxyas force-pushed the ignore-allowed-lints-final branch from 001f225 to 718243c Compare May 19, 2024 14:28

Xiretza reviewed May 19, 2024

View reviewed changes

compiler/rustc_lint/src/late.rs Outdated Show resolved Hide resolved

compiler/rustc_middle/src/query/mod.rs Outdated Show resolved Hide resolved

compiler/rustc_parse/src/parser/attr.rs Outdated Show resolved Hide resolved

faptc reviewed May 19, 2024

View reviewed changes

compiler/rustc_lint/src/late.rs Outdated Show resolved Hide resolved

compiler/rustc_lint/src/late.rs Outdated Show resolved Hide resolved

blyxyas force-pushed the ignore-allowed-lints-final branch from fdd588d to 67d48d7 Compare May 26, 2024 22:27

This comment has been minimized.

Sign in to view

michaelwoerister reviewed May 27, 2024

View reviewed changes

blyxyas added 4 commits May 27, 2024 13:59

Fix nits from review

5d46f58

name_without_tool Now takes a &str instead of String

20e1f87

Change lints_to_emit to lints_that_actually_run

16d8d49

blyxyas force-pushed the ignore-allowed-lints-final branch from 6b203f9 to 16d8d49 Compare May 27, 2024 12:00

rust-lang deleted a comment from rust-log-analyzer May 27, 2024

Fix CI, and use FxIndexSet

7b53e10

blyxyas marked this pull request as draft May 27, 2024 18:42

blyxyas marked this pull request as ready for review May 27, 2024 19:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(Big performance change) Do not run lints that cannot emit #125116

(Big performance change) Do not run lints that cannot emit #125116

blyxyas commented May 14, 2024 •

edited

rustbot commented May 14, 2024

rustbot commented May 14, 2024

blyxyas commented May 14, 2024

matthiaskrgr commented May 14, 2024

This comment has been minimized.

bors commented May 14, 2024

This comment has been minimized.

Kobzol commented May 14, 2024

bors commented May 14, 2024

This comment has been minimized.

lqd commented May 14, 2024 •

edited

rust-timer commented May 14, 2024

Centri3 commented May 14, 2024 •

edited

blyxyas commented May 15, 2024 •

edited

blyxyas commented May 15, 2024

blyxyas commented May 15, 2024

Kobzol commented May 15, 2024

blyxyas commented May 15, 2024

This comment has been minimized.

bors commented May 15, 2024

This comment has been minimized.

blyxyas commented May 18, 2024

Manishearth commented May 19, 2024

This comment has been minimized.

Xiretza left a comment

michaelwoerister commented May 21, 2024

blyxyas commented May 21, 2024

Manishearth commented May 21, 2024

This comment has been minimized.

michaelwoerister May 27, 2024

michaelwoerister May 27, 2024

michaelwoerister commented May 27, 2024

michaelwoerister May 27, 2024 •

edited

bors commented May 27, 2024

(Big performance change) Do not run lints that cannot emit #125116

Are you sure you want to change the base?

(Big performance change) Do not run lints that cannot emit #125116

Conversation

blyxyas commented May 14, 2024 • edited

rustbot commented May 14, 2024

rustbot commented May 14, 2024

blyxyas commented May 14, 2024

matthiaskrgr commented May 14, 2024

This comment has been minimized.

bors commented May 14, 2024

This comment has been minimized.

Kobzol commented May 14, 2024

bors commented May 14, 2024

This comment has been minimized.

lqd commented May 14, 2024 • edited

rust-timer commented May 14, 2024

Overall result: ❌ regressions - ACTION NEEDED

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Centri3 commented May 14, 2024 • edited

blyxyas commented May 15, 2024 • edited

blyxyas commented May 15, 2024

Wall time

Max RSS

Instructions

Cycles

blyxyas commented May 15, 2024

Kobzol commented May 15, 2024

blyxyas commented May 15, 2024

This comment has been minimized.

bors commented May 15, 2024

This comment has been minimized.

blyxyas commented May 18, 2024

Manishearth commented May 19, 2024

This comment has been minimized.

Xiretza left a comment

Choose a reason for hiding this comment

michaelwoerister commented May 21, 2024

blyxyas commented May 21, 2024

Manishearth commented May 21, 2024

This comment has been minimized.

michaelwoerister May 27, 2024

Choose a reason for hiding this comment

michaelwoerister May 27, 2024

Choose a reason for hiding this comment

michaelwoerister commented May 27, 2024

michaelwoerister May 27, 2024 • edited

Choose a reason for hiding this comment

bors commented May 27, 2024

blyxyas commented May 14, 2024 •

edited

lqd commented May 14, 2024 •

edited

Centri3 commented May 14, 2024 •

edited

blyxyas commented May 15, 2024 •

edited

michaelwoerister May 27, 2024 •

edited