Parallel lints

[jw013]

Closes #621.

The rayon implementation lacks the per query "Running {query}\r" dynamic messages as I couldn't think of any sensible way to do that with multiple threads.

[obi1kenobi]

Thanks, this is great!

Any chance we can tackle this portion of the linked issue in the process as well?

In particular, ensure that reported timing information accurately communicates how much time is spent generating rustdoc data (serial, blocking) and how much time is spent actually running each lint (after this change, parallel across lints) — this is covered by #298.

[jw013]

Sure, I can have a go at #298 this next week. When I do, do you want me to make a new PR branch or just add to this one?

[obi1kenobi]

Let's go for a new branch, just to make it easy to review. If you'd like, feel free to put together a draft of what the output could look like before taking the time to implement it. There are some edge cases to consider (e.g. what if there are multiple crates to check in the workspace, etc. with other CLI flags) that could be good to work out "on paper" before investing into an implementation. Similar edge cases come up when reporting parallel timing as well, which is why I suggested tackling these two issues together.

…

On Mon, Jan 8, 2024, 12:09 AM jw013 ***@***.***> wrote: Sure, I can have a go at #298 <#298> this next week. When I do, do you want me to make a new PR branch or just add to this one? — Reply to this email directly, view it on GitHub <#625 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAR5MSVWKXHHB7XVQK7NDZDYNN5PTAVCNFSM6AAAAABBQZBTFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBQGM4TENZTGA> . You are receiving this because you commented.Message ID: ***@***.***>

[obi1kenobi]

One nitpick, otherwise this is good to go! 🚀

Thanks for putting it together!

[obi1kenobi]

Based on an unscientific "I compared two CI runs of one of one of our beefier tests," GitHub Actions runners have 4 cores and the speedup is in the neighborhood of 2-2.5x. This would make sense if they have 2 physical cores with hyperthreading enabled.

I'll do some more significant tests before we ship the next release.

[obi1kenobi]

Awesome! 🚀

I'm planning to post about this PR on social media, and I'd like to give you a shout-out if you're okay with that (no pressure if not!). Lmk if you have a Mastodon / Twitter / bsky I should tag in the post?

[jw013]

Awesome! 🚀

I'm planning to post about this PR on social media, and I'd like to give you a shout-out if you're okay with that (no pressure if not!). Lmk if you have a Mastodon / Twitter / bsky I should tag in the post?

That's fine with me. Outside of lurking occasionally on Mastodon I don't really use social media.

[obi1kenobi]

Happy to tag your Mastodon if you'd like (just let me know what it is). Again, no pressure!

[obi1kenobi]

Also, if you're interested in contributing more things, happy to help however I can! There's a mix of issues on the issue tracker, but I haven't done an amazing job of curating them thus far so feel free to ping me with any questions or ask for suggestions.

[jw013]

Mastodon is how I found out about this project - I only use it to follow people who post interesting things about Rust and other topics, so probably not worth mentioning.

I'm happy just picking through open issues for now - what I have seen so far has been well documented and fairly easy to understand so I definitely appreciate that plus the additional feedback. Thanks!