Documentation feature request - Rule performance benchmarks

[CreativeTechGuy]

Add performance profiling benchmarks for each rule and display these benchmark numbers on the rule summary page. (Maybe a relative score or a green/yellow/red indicator.)

Motivation:

It may not be obvious when first enabling a rule the performance impact it'll have in the long run. It'd help users make decisions about which rules are worth adding depending on how slow the rules are to evaluate. For example: if a rule is a very fringe case that may never apply to your codebase but has a very high performance impact, you would probably choose to not enable it. It'd hopefully also help reduce the number of "it's too slow" Issues being opened.

From a Rule Developer's perspective it'd be a great way to see if the rule you've created is performant or not. There's already not great documentation on how to write rules and it can be easy to find something that functions as you intended but turns out it's extremely slow and you wouldn't realize it until you try it on a massive codebase.

Implementation:

There's already debugging and profiling information, the missing piece is the large and varied codebase to run it on and automatically collect the stats. We might be able to find several other large open-source projects that use TypeScript heavily and run each rule against that. Having enough different codebases can produce a range of values for the performance of each rule. As a step of generating documentation, the command could automatically run the rule against all of those massive projects and collect the stats.

[Josh-Cena]

This is quite hard because it's practically impossible to benchmark rules in isolation. See for example #5495, #8189 (comment). It doesn't mean we are not interested in what rules are spending time; we will pursue #8087 instead where we can examine things more holistically.