New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clippy 1.0 #2476

Open
wants to merge 15 commits into
base: master
from

Conversation

Projects
None yet
@Manishearth
Member

Manishearth commented Jun 14, 2018

This RFC proposes a Clippy 1.0 in preparation for being shipped via Rustup. It aims to get community consensus on the current state of Clippy lints, as well as on what lints we should uplift to the compiler directly if any.

See also: The Future of Clippy

Co-written by @oli-obk

Rendered

Manishearth and others added some commits Jun 14, 2018

Merge pull request #2 from oli-obk/patch-2
Update 0000-clippy-uno.md
@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jun 14, 2018

Member

cc @rust-lang/dev-tools @rust-lang/compiler

This isn't tagged T-Compiler (maybe it should be?) but the uplift stuff does impact the compiler team.

Member

Manishearth commented Jun 14, 2018

cc @rust-lang/dev-tools @rust-lang/compiler

This isn't tagged T-Compiler (maybe it should be?) but the uplift stuff does impact the compiler team.

Show outdated Hide outdated text/0000-clippy-uno.md
Show outdated Hide outdated text/0000-clippy-uno.md
# Rationale and alternatives
[alternatives]: #alternatives
We don't particularly _need_ a 1.0, however it's good to have a milestone here, and a general idea of stability as we move forward in this process.

This comment has been minimized.

@mcarton

mcarton Jun 14, 2018

I once gave a talk about clippy and the first question asked by the audience was "Why the fuck is your version 0.0.97? Are you ever going to increase from 0.0?". This was more than 100 versions ago.
We need to move away from 0.0 at some point.

@mcarton

mcarton Jun 14, 2018

I once gave a talk about clippy and the first question asked by the audience was "Why the fuck is your version 0.0.97? Are you ever going to increase from 0.0?". This was more than 100 versions ago.
We need to move away from 0.0 at some point.

This comment has been minimized.

@Victor-Savu

Victor-Savu Jun 22, 2018

Such an angry audience you had :) I hope they are not representative for the rust community.

@Victor-Savu

Victor-Savu Jun 22, 2018

Such an angry audience you had :) I hope they are not representative for the rust community.

This comment has been minimized.

@mcarton

mcarton Jun 23, 2018

They weren't particularly angry, just really astonished by such a weird version. I also mentioned semver in a previous talk about Rust at the same conference, and then here I am, presenting a tool that seems like it doesn't want to commit to a proper versioning. Kind of ironic for a tool that aims to help you with Rust good practises.

@mcarton

mcarton Jun 23, 2018

They weren't particularly angry, just really astonished by such a weird version. I also mentioned semver in a previous talk about Rust at the same conference, and then here I am, presenting a tool that seems like it doesn't want to commit to a proper versioning. Kind of ironic for a tool that aims to help you with Rust good practises.

Show outdated Hide outdated text/0000-clippy-uno.md
@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jun 14, 2018

Member

cc the other clippy maintainers @llogiq @mcarton @phansch @flip1995 @birkenfeld

Member

Manishearth commented Jun 14, 2018

cc the other clippy maintainers @llogiq @mcarton @phansch @flip1995 @birkenfeld

@clarcharr

This comment has been minimized.

Show comment
Hide comment
@clarcharr

clarcharr Jun 14, 2018

Contributor

A few thoughts:

  1. How should clippy deal with stability of lints that recommend using an external crate? For example, the naive_bytecount lint will redirect the user to the bytecount crate. If this crate updates to a new major version (not likely in this case, but in general), how should the clippy lint respond?
  2. I recall somewhere there being a style guide for lint names for rustc, and I think that such a style should be adopted for clippy too. The names of the lints seem very inconsistent and all over the place, and it makes sense to standardise them before 1.0.
  3. There are a few lints that abbreviate words that I think should not. For example, cmp_owned might be better as owned_comparison rather than deferring to the cmp abbreviation used by Ord and PartialOrd. Similarly, manual_memcpy should be manual_slice_copy IMO because it talks about what's done, rather than deferring to the term memcpy which is again a function name.

Also:

  • unused_collect may be replaceable with a #[must_use] on collect? Not 100% sure but that seems to make sense to me.
Contributor

clarcharr commented Jun 14, 2018

A few thoughts:

  1. How should clippy deal with stability of lints that recommend using an external crate? For example, the naive_bytecount lint will redirect the user to the bytecount crate. If this crate updates to a new major version (not likely in this case, but in general), how should the clippy lint respond?
  2. I recall somewhere there being a style guide for lint names for rustc, and I think that such a style should be adopted for clippy too. The names of the lints seem very inconsistent and all over the place, and it makes sense to standardise them before 1.0.
  3. There are a few lints that abbreviate words that I think should not. For example, cmp_owned might be better as owned_comparison rather than deferring to the cmp abbreviation used by Ord and PartialOrd. Similarly, manual_memcpy should be manual_slice_copy IMO because it talks about what's done, rather than deferring to the term memcpy which is again a function name.

Also:

  • unused_collect may be replaceable with a #[must_use] on collect? Not 100% sure but that seems to make sense to me.
@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jun 14, 2018

Member

If this crate updates to a new major version (not likely in this case, but in general), how should the clippy lint respond?

Still redirect them to the crate if the crate still works the same way. If not, remove the lint.

The names of the lints seem very inconsistent and all over the place, and it makes sense to standardise them before 1.0.
...
There are a few lints that abbreviate words that I think should not

Would you like to come up with a list of potential renames? 😄 It seems like you have a rough idea of a consistent system here. We can make this happen; and it doesn't need to happen in-band in this RFC (we can cross link a PR). It's in scope for this RFC but I want to break out discussions as much as possible.

Edit: Discussion at rust-lang-nursery/rust-clippy#2845

unused_collect may be replaceable with a #[must_use] on collect? Not 100% sure but that seems to make sense to me.

That's a fair point, could you open a rustc PR for that? We can deprecate the lint if it ends up landing. But that's out of scope for this RFC so I don't want to discuss it here too much 😄

Edit: Moved to rust-lang-nursery/rust-clippy#2846

Member

Manishearth commented Jun 14, 2018

If this crate updates to a new major version (not likely in this case, but in general), how should the clippy lint respond?

Still redirect them to the crate if the crate still works the same way. If not, remove the lint.

The names of the lints seem very inconsistent and all over the place, and it makes sense to standardise them before 1.0.
...
There are a few lints that abbreviate words that I think should not

Would you like to come up with a list of potential renames? 😄 It seems like you have a rough idea of a consistent system here. We can make this happen; and it doesn't need to happen in-band in this RFC (we can cross link a PR). It's in scope for this RFC but I want to break out discussions as much as possible.

Edit: Discussion at rust-lang-nursery/rust-clippy#2845

unused_collect may be replaceable with a #[must_use] on collect? Not 100% sure but that seems to make sense to me.

That's a fair point, could you open a rustc PR for that? We can deprecate the lint if it ends up landing. But that's out of scope for this RFC so I don't want to discuss it here too much 😄

Edit: Moved to rust-lang-nursery/rust-clippy#2846

@Diggsey

This comment was marked as resolved.

Show comment
Hide comment
@Diggsey

Diggsey Jun 15, 2018

Contributor

The lint categorisation section appears to have the same set of lints for all categories?

Contributor

Diggsey commented Jun 15, 2018

The lint categorisation section appears to have the same set of lints for all categories?

@F001

This comment was marked as resolved.

Show comment
Hide comment
@F001

F001 Jun 15, 2018

Contributor

I have a question that why it is named as "clippy". Why not just "rustlint" or something like that?

Contributor

F001 commented Jun 15, 2018

I have a question that why it is named as "clippy". Why not just "rustlint" or something like that?

@Centril

This comment has been minimized.

Show comment
Hide comment
@Centril

Centril Jun 15, 2018

Contributor

This RFC does not yet propose lints to be uplifted, but the intention is that the RFC discussion will bring up lints that the community feels should be uplifted and we can list them here.

Apologies in advance if this is a major land-grab, but since uplifting to rustc is being discussed here and that compiler lints are under the purview of the lang team, I'm also adding T-lang.

@F001

I have a question that why it is named as "clippy". Why not just "rustlint" or something like that?

In honour of Clippy, the Office Assistant. I think it is pretty cute.

Contributor

Centril commented Jun 15, 2018

This RFC does not yet propose lints to be uplifted, but the intention is that the RFC discussion will bring up lints that the community feels should be uplifted and we can list them here.

Apologies in advance if this is a major land-grab, but since uplifting to rustc is being discussed here and that compiler lints are under the purview of the lang team, I'm also adding T-lang.

@F001

I have a question that why it is named as "clippy". Why not just "rustlint" or something like that?

In honour of Clippy, the Office Assistant. I think it is pretty cute.

@Centril Centril added the T-lang label Jun 15, 2018

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jun 15, 2018

Member

I feel like lints fall under the purview of the compiler team; at least this discussion has been ongoing with the compiler team, not the lang team. (I didn't include the compiler team myself because I wanted them to make the decision of adding themselves)

Lints are a UI feature, not a language feature. Alternative implementations need not use the same set of lints.

Member

Manishearth commented Jun 15, 2018

I feel like lints fall under the purview of the compiler team; at least this discussion has been ongoing with the compiler team, not the lang team. (I didn't include the compiler team myself because I wanted them to make the decision of adding themselves)

Lints are a UI feature, not a language feature. Alternative implementations need not use the same set of lints.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jun 15, 2018

Member

The lint categorisation section appears to have the same set of lints for all categories?

Cc @oli-obk bug in your script? :)

Member

Manishearth commented Jun 15, 2018

The lint categorisation section appears to have the same set of lints for all categories?

Cc @oli-obk bug in your script? :)

@Centril

This comment has been minimized.

Show comment
Hide comment
@Centril

Centril Jun 15, 2018

Contributor

@Manishearth Hmm well; It could be that the RFC policy - language design document is outdated, but just today we did discuss #2471 in the lang team meeting.

My rationale for continued T-lang purview (in addition to T-dev-tools) is mainly that while lints are a UI feature, as you say, compiler lints are important wrt. the feel of the language itself, so they factor into the design of the language. Of course, by that logic, major swaths of T-libs responsibilities (and of other teams..) could suddenly become T-lang, but you have to draw the line somewhere, so I draw the line at compiler lints.

Anyways, if the language team agrees this is not under their purview they can remove themselves... ;)
If that happens, we should update the lang_changes.md file in some way.

EDIT: nominating this for the next lang meeting so the question of purview itself can be dealt with quickly.

Contributor

Centril commented Jun 15, 2018

@Manishearth Hmm well; It could be that the RFC policy - language design document is outdated, but just today we did discuss #2471 in the lang team meeting.

My rationale for continued T-lang purview (in addition to T-dev-tools) is mainly that while lints are a UI feature, as you say, compiler lints are important wrt. the feel of the language itself, so they factor into the design of the language. Of course, by that logic, major swaths of T-libs responsibilities (and of other teams..) could suddenly become T-lang, but you have to draw the line somewhere, so I draw the line at compiler lints.

Anyways, if the language team agrees this is not under their purview they can remove themselves... ;)
If that happens, we should update the lang_changes.md file in some way.

EDIT: nominating this for the next lang meeting so the question of purview itself can be dealt with quickly.

@Centril Centril added the I-nominated label Jun 15, 2018

[online]: http://rust-lang-nursery.github.io/rust-clippy/current/
Please leave comments on thoughts about these lints -- if their categorization is correct, if they should exist at all, and if we should be uplifting them to the compiler.

This comment was marked as resolved.

@scottmcm

scottmcm Jun 15, 2018

Member

It looks like every single section in this RFC has the same lints? Copy-paste error?

@scottmcm

scottmcm Jun 15, 2018

Member

It looks like every single section in this RFC has the same lints? Copy-paste error?

This comment was marked as outdated.

@birkenfeld

birkenfeld Jun 15, 2018

And there was I commenting that all those lints belong to perf...

@birkenfeld

birkenfeld Jun 15, 2018

And there was I commenting that all those lints belong to perf...

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jun 15, 2018

Member

I think some lints are lang team things, but not in general. I consider that document outdated because Rust has had a semi official "no new lints" policy for ages which effectively means it never got exercised, and historically we've added lints along with language changes to support them, or without an RFC at all in case of things like soundness deprecations.

Compiler lints are as important as diagnostics in the feel of the language imo, but diagnostics are squarely a compiler team concern. Docs are also in a similar space here. Ultimately everything we do here impacts Rust the language 😄


I basically want to avoid this becoming a three team RFC 😄

And also, this is something we've been discussing with the compiler team for a while which is why I want to avoid additional churn and rehashing things we've discussed.

Member

Manishearth commented Jun 15, 2018

I think some lints are lang team things, but not in general. I consider that document outdated because Rust has had a semi official "no new lints" policy for ages which effectively means it never got exercised, and historically we've added lints along with language changes to support them, or without an RFC at all in case of things like soundness deprecations.

Compiler lints are as important as diagnostics in the feel of the language imo, but diagnostics are squarely a compiler team concern. Docs are also in a similar space here. Ultimately everything we do here impacts Rust the language 😄


I basically want to avoid this becoming a three team RFC 😄

And also, this is something we've been discussing with the compiler team for a while which is why I want to avoid additional churn and rehashing things we've discussed.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jun 15, 2018

Member

Updated the lint list, sorry about that.

Member

Manishearth commented Jun 15, 2018

Updated the lint list, sorry about that.

the result.
- [expect_fun_call](https://rust-lang-nursery.github.io/rust-clippy/master/index.html#expect_fun_call): Checks for calls to `.expect(&format!(...))`, `.expect(foo(..))`,
etc., and suggests to use `unwrap_or_else` instead
- [naive_bytecount](https://rust-lang-nursery.github.io/rust-clippy/master/index.html#naive_bytecount): Checks for naive byte counts

This comment has been minimized.

@Manishearth

Manishearth Jun 15, 2018

Member

Moving over @scottmcm's comment since it got hidden:

It feels weird to me for an official rust-lang-organization tool to ever recommend a non-rust-lang crate in a lint, no matter how good it is. I never want to see an issue for "my crate for this is better than that one, so you should be recommending mine instead".

@Manishearth

Manishearth Jun 15, 2018

Member

Moving over @scottmcm's comment since it got hidden:

It feels weird to me for an official rust-lang-organization tool to ever recommend a non-rust-lang crate in a lint, no matter how good it is. I never want to see an issue for "my crate for this is better than that one, so you should be recommending mine instead".

This comment has been minimized.

@Manishearth

Manishearth Jun 15, 2018

Member

I feel like it's fine for us to do this; we also have some lints about serde (they don't suggest serde however). It's currently the only crate for this task, used by ripgrep, so okay to suggest. If this ever becomes contentious we can reevaluate; removing lints via deprecation isn't an issue.

There's a slippery slope argument to be made here, though; what if other such lints crop up.

@Manishearth

Manishearth Jun 15, 2018

Member

I feel like it's fine for us to do this; we also have some lints about serde (they don't suggest serde however). It's currently the only crate for this task, used by ripgrep, so okay to suggest. If this ever becomes contentious we can reevaluate; removing lints via deprecation isn't an issue.

There's a slippery slope argument to be made here, though; what if other such lints crop up.

- [serde_api_misuse](https://rust-lang-nursery.github.io/rust-clippy/master/index.html#serde_api_misuse): Checks for mis-uses of the serde API.
- [forget_ref](https://rust-lang-nursery.github.io/rust-clippy/master/index.html#forget_ref): Checks for calls to `std::mem::forget` with a reference
instead of an owned value.
- [absurd_extreme_comparisons](https://rust-lang-nursery.github.io/rust-clippy/master/index.html#absurd_extreme_comparisons): Checks for comparisons where one side of the relation is

This comment has been minimized.

@Manishearth

Manishearth Jun 15, 2018

Member

This lint is the same as unused_comparisons lint in rustc, however our implementation handles a bunch of edge cases and detects a couple more things like boolean cases. Perhaps we should deprecate the clippy lint but uplift the implementation.

(credit @clarcharr for noticing this)

@Manishearth

Manishearth Jun 15, 2018

Member

This lint is the same as unused_comparisons lint in rustc, however our implementation handles a bunch of edge cases and detects a couple more things like boolean cases. Perhaps we should deprecate the clippy lint but uplift the implementation.

(credit @clarcharr for noticing this)

This comment has been minimized.

@Manishearth

Manishearth Aug 31, 2018

Member

This is being uplifted in rust-lang/rust#53224 , so we don't have to deal with it here

@Manishearth

Manishearth Aug 31, 2018

Member

This is being uplifted in rust-lang/rust#53224 , so we don't have to deal with it here

@mark-i-m

This comment has been minimized.

Show comment
Hide comment
@mark-i-m

mark-i-m Jun 16, 2018

Contributor

Bikeshed: cargo lint instead of cargo lint. Mostly because it's shorter to type and a more intuitive for new users.

Contributor

mark-i-m commented Jun 16, 2018

Bikeshed: cargo lint instead of cargo lint. Mostly because it's shorter to type and a more intuitive for new users.

@Centril

This comment has been minimized.

Show comment
Hide comment
@Centril

Centril Jun 16, 2018

Contributor

@mark-i-m Counter-bikeshed: clippy (in addition to cargo clippy) is even shorter ;)

I think it would be reasonable to add an alias cargo lint if someone wants that; but I think the name should be clippy because of the cuteness factor.

Contributor

Centril commented Jun 16, 2018

@mark-i-m Counter-bikeshed: clippy (in addition to cargo clippy) is even shorter ;)

I think it would be reasonable to add an alias cargo lint if someone wants that; but I think the name should be clippy because of the cuteness factor.

@leodasvacas

This comment has been minimized.

Show comment
Hide comment
@leodasvacas

leodasvacas Jun 19, 2018

explicit_iter_loop - This one is controversial, some find .iter() or .iter_mut() to be better style.

cyclomatic_complexity and too_many_arguments - These ones I find pedantic, because they are difficult to fix and many times the code is intentionally complex rather than accidentally complex.

leodasvacas commented Jun 19, 2018

explicit_iter_loop - This one is controversial, some find .iter() or .iter_mut() to be better style.

cyclomatic_complexity and too_many_arguments - These ones I find pedantic, because they are difficult to fix and many times the code is intentionally complex rather than accidentally complex.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jun 19, 2018

Member

@leodasvacas I'm surprised with the former, is this really the case for many? We should probably establish a style decision here.

For the latter, yes, they are difficult to fix, but even "intentionally complex" code can be split into functions, and in the rare cases it can't you should probably refactor. "hard to fix" does not make something not-clippy-worthy, you can always silence the lint if you want the easy way out.

(Also please leave comments inline)

Member

Manishearth commented Jun 19, 2018

@leodasvacas I'm surprised with the former, is this really the case for many? We should probably establish a style decision here.

For the latter, yes, they are difficult to fix, but even "intentionally complex" code can be split into functions, and in the rare cases it can't you should probably refactor. "hard to fix" does not make something not-clippy-worthy, you can always silence the lint if you want the easy way out.

(Also please leave comments inline)

be replaced by safe conversion functions.
- [unnecessary_operation](https://rust-lang-nursery.github.io/rust-clippy/master/index.html#unnecessary_operation): Checks for expression statements that can be reduced to a
sub-expression.
- [cyclomatic_complexity](https://rust-lang-nursery.github.io/rust-clippy/master/index.html#cyclomatic_complexity): Checks for methods with high cyclomatic complexity.

This comment has been minimized.

@fhartwig

fhartwig Jun 20, 2018

Cyclomatic complexity is one of the best examples of Goodhart's law, meaning that high cyclomatic complexity might be a sign of hard-to-understand code, but intentionally trying to reduce it is almost certainly going to make it even worse. As someone else once pointed out to me, you can reduce cyclomatic complexity to 1 by Church-encoding all conditionals and replacing all loops by recursion. Which does not improve readability.

So I think this lint should be allow by default.

@fhartwig

fhartwig Jun 20, 2018

Cyclomatic complexity is one of the best examples of Goodhart's law, meaning that high cyclomatic complexity might be a sign of hard-to-understand code, but intentionally trying to reduce it is almost certainly going to make it even worse. As someone else once pointed out to me, you can reduce cyclomatic complexity to 1 by Church-encoding all conditionals and replacing all loops by recursion. Which does not improve readability.

So I think this lint should be allow by default.

This comment has been minimized.

@oli-obk

oli-obk Jun 20, 2018

Contributor

replacing all loops by recursion. Which does not improve readability.

This statement probably makes a lot of haskellers very angry 😛

On a more serious note:

There is almost always a bad way to resolve any lint. The correct solution to the CC lint triggering in case there is no way to nicely refactor the function is to increase the limit for the function.

What I have noticed is that high CC functions can often be split up into multiple ones which, instead of having everything in scope, just have a few arguments and often just one or two return values. Increasing modularity is a good thing in my book. Ideally the CC lint would be able to suggest good spots for refactoring out code into separate functions.

Note that the CC lint is very Rust-tuned and not just the straight forward original implementation. This is due to the fact that Rust uses match extensively (which in vanilla CC has a CC of the number of arms + the CC of all arm bodies). Other languages usually introduce a single dynamic dispatch call (which has a CC of 1). Thus the clippy-CC treats a match as having a CC of just its arms' bodies CC. So if all the arm bodies have a CC of 1, the entire match has a CC of 2 (to account for some branching going on).

The same goes for short circuting operations (&& and ||), these do not increase the CC.

@oli-obk

oli-obk Jun 20, 2018

Contributor

replacing all loops by recursion. Which does not improve readability.

This statement probably makes a lot of haskellers very angry 😛

On a more serious note:

There is almost always a bad way to resolve any lint. The correct solution to the CC lint triggering in case there is no way to nicely refactor the function is to increase the limit for the function.

What I have noticed is that high CC functions can often be split up into multiple ones which, instead of having everything in scope, just have a few arguments and often just one or two return values. Increasing modularity is a good thing in my book. Ideally the CC lint would be able to suggest good spots for refactoring out code into separate functions.

Note that the CC lint is very Rust-tuned and not just the straight forward original implementation. This is due to the fact that Rust uses match extensively (which in vanilla CC has a CC of the number of arms + the CC of all arm bodies). Other languages usually introduce a single dynamic dispatch call (which has a CC of 1). Thus the clippy-CC treats a match as having a CC of just its arms' bodies CC. So if all the arm bodies have a CC of 1, the entire match has a CC of 2 (to account for some branching going on).

The same goes for short circuting operations (&& and ||), these do not increase the CC.

This comment has been minimized.

@Centril

Centril Jun 20, 2018

Contributor

This statement probably makes a lot of haskellers very angry 😛

More like: Not angry, just disappointed ;)

Tho, once you've done some manual recursion a few times, you start using folds and maps and stuff and even writer, reader, and state monads. There are many ways to be a Haskell programmer.. See: The Evolution of a Haskell Programmer.

I think your observation is spot on otherwise. Using more higher order functions is also a nice way to get rid of rightward drift and cyclomatic complexity, or a more compositional style in general. The worst thing you can have is long and deep functions; Long and shallow functions are not the worst, but also not very great. Short and shallow gives you nice readability.

@Centril

Centril Jun 20, 2018

Contributor

This statement probably makes a lot of haskellers very angry 😛

More like: Not angry, just disappointed ;)

Tho, once you've done some manual recursion a few times, you start using folds and maps and stuff and even writer, reader, and state monads. There are many ways to be a Haskell programmer.. See: The Evolution of a Haskell Programmer.

I think your observation is spot on otherwise. Using more higher order functions is also a nice way to get rid of rightward drift and cyclomatic complexity, or a more compositional style in general. The worst thing you can have is long and deep functions; Long and shallow functions are not the worst, but also not very great. Short and shallow gives you nice readability.

This comment has been minimized.

@fhartwig

fhartwig Jun 20, 2018

The worst thing you can have is long and deep functions; Long and shallow functions are not the worst, but also not very great. Short and shallow gives you nice readability.

This is sometimes true, but not universally. I'm a Haskell programmer too, and I've seen some absolutely unreadable short and shallow functions that overused higher-order functions and point-free style, whose readability would have greatly improved by not code-golfing and being overly clever.

Which is really my point. CC is an incredibly simplistic heuristic for something that is inherently difficult to measure, and trying to optimise your code specifically for it will make it worse, rather than better. The programmer needs to consider trade-offs when it comes to readability, for instance the trade-off between rightward-drift and extra indirection. This requires good judgment. I'd really prefer if the warn-by-default lints would stick to recommendations that make your code unambiguously better (like 'use a byte literal rather than casting char literals to bytes').

@fhartwig

fhartwig Jun 20, 2018

The worst thing you can have is long and deep functions; Long and shallow functions are not the worst, but also not very great. Short and shallow gives you nice readability.

This is sometimes true, but not universally. I'm a Haskell programmer too, and I've seen some absolutely unreadable short and shallow functions that overused higher-order functions and point-free style, whose readability would have greatly improved by not code-golfing and being overly clever.

Which is really my point. CC is an incredibly simplistic heuristic for something that is inherently difficult to measure, and trying to optimise your code specifically for it will make it worse, rather than better. The programmer needs to consider trade-offs when it comes to readability, for instance the trade-off between rightward-drift and extra indirection. This requires good judgment. I'd really prefer if the warn-by-default lints would stick to recommendations that make your code unambiguously better (like 'use a byte literal rather than casting char literals to bytes').

This comment has been minimized.

@Manishearth

Manishearth Jun 20, 2018

Member

I'd really prefer if the allow-by-default lints would stick to recommendations that make your code unambiguously better

Did you mean warn-by-default? (cyclomatic is warn but I see a good argument here for making it allow in the pedantic group)

@Manishearth

Manishearth Jun 20, 2018

Member

I'd really prefer if the allow-by-default lints would stick to recommendations that make your code unambiguously better

Did you mean warn-by-default? (cyclomatic is warn but I see a good argument here for making it allow in the pedantic group)

This comment has been minimized.

@fhartwig

fhartwig Jun 20, 2018

@Manishearth I did mean warn-by-default, thanks for pointing that out!

@fhartwig

fhartwig Jun 20, 2018

@Manishearth I did mean warn-by-default, thanks for pointing that out!

This comment has been minimized.

@Centril

Centril Jun 20, 2018

Contributor

This is sometimes true, but not universally.

Of course. :) Very few things are universally true. In this case I'd say it is more often true than not.

I'm a Haskell programmer too,

👋 ❤️

I've seen some absolutely unreadable short and shallow functions that overused higher-order functions and point-free style, whose readability would have greatly improved by not code-golfing and being overly clever.

Sure; if you write code looking like it was processed through http://pointfree.io/, then it will look unreadable. Particularly if you combine flip and sectioning of (.). However, using function composition linearly (where associativity applies) to a bunch of small functions makes code both more maintainable, reusable and readable. How readable such code is also depends on how many combinators you have in your toolbox. Of course, you need to strike a balance between pointfree style and pointfull style. A few well chosen places to use a lambda here and there can improve readability. And while the Haskeller who has entered their rigorous phase (according to themselves), and who have started to grasp the more advanced category-theoretical concepts, might eschew do notation, a post-rigorous Haskeller will probably enjoy using it.

Something I find particularly nice about shallow and short functions is that you also get more type signatures (assuming you put type signatures on all top level functions, which you should), which helps understanding (due to free theorems) in my opinion. Functions also become more readily testable with QuickCheck thus improving the reliability of your code as a consequence. I also personally tend to document every single top level function I write with haddock in Haskell. As a consequence, even if you don't understand the details of the small function, you can more easily grasp the overall semantics and the big picture. Having the habit of writing short functions also is a boon for recognizing patterns and extracting out common algorithms.

If you want to compare styles, here is a code style document I wrote for my BsC thesis project.

Contrast this to long and deep functions with a bunch of nested where blocks and case expressions. Then you get fewer type signatures, logic that is harder to test, and less reuse.

Also, if we consider a language such as Agda or Idris, splitting proof obligations out to a different function can make development much easier by having fewer things to deal with in your short term memory. Edwin Brady demonstrates this in his talk on Type-driven Development with Idris

Which is really my point. CC is an incredibly simplistic heuristic for something that is inherently difficult to measure, and trying to optimize your code specifically for it will make it worse, rather than better.

While being a simple heuristic as you say, I think it often hits well. In particular, my view is that a lot of Rust code already tends to involve too many long and deep functions (spanning 300 lines) and very long files that make it hard (at least for me) to gauge the overall picture. I think modules, as a consequence, are too underused in Rust. As such, I would much prefer to retain the warn-by-default nature of the cyclomatic_complexity lint in the complexity group.

The programmer needs to consider trade-offs when it comes to readability, for instance the trade-off between rightward-drift and extra indirection. This requires good judgment.

It does. But I find that it also depends on how well the out-extracted function is named. A small and short function calling other aptly named small and short functions can read like pseudo-code. A good IDE can also enable you to quickly jump to the definition of the helper functions. Therefore, I think the default setting should be to write short and sweet definitions.

I'd really prefer if the warn-by-default lints would stick to recommendations that make your code unambiguously better (like 'use a byte literal rather than casting char literals to bytes').

For the reasons I've outlined above, I disagree. To me, lints such as too_many_arguments, type_complexity, and cyclomatic_complexity induce good habits, even if I've been hit by them myself and felt like it was unwarranted.

@Centril

Centril Jun 20, 2018

Contributor

This is sometimes true, but not universally.

Of course. :) Very few things are universally true. In this case I'd say it is more often true than not.

I'm a Haskell programmer too,

👋 ❤️

I've seen some absolutely unreadable short and shallow functions that overused higher-order functions and point-free style, whose readability would have greatly improved by not code-golfing and being overly clever.

Sure; if you write code looking like it was processed through http://pointfree.io/, then it will look unreadable. Particularly if you combine flip and sectioning of (.). However, using function composition linearly (where associativity applies) to a bunch of small functions makes code both more maintainable, reusable and readable. How readable such code is also depends on how many combinators you have in your toolbox. Of course, you need to strike a balance between pointfree style and pointfull style. A few well chosen places to use a lambda here and there can improve readability. And while the Haskeller who has entered their rigorous phase (according to themselves), and who have started to grasp the more advanced category-theoretical concepts, might eschew do notation, a post-rigorous Haskeller will probably enjoy using it.

Something I find particularly nice about shallow and short functions is that you also get more type signatures (assuming you put type signatures on all top level functions, which you should), which helps understanding (due to free theorems) in my opinion. Functions also become more readily testable with QuickCheck thus improving the reliability of your code as a consequence. I also personally tend to document every single top level function I write with haddock in Haskell. As a consequence, even if you don't understand the details of the small function, you can more easily grasp the overall semantics and the big picture. Having the habit of writing short functions also is a boon for recognizing patterns and extracting out common algorithms.

If you want to compare styles, here is a code style document I wrote for my BsC thesis project.

Contrast this to long and deep functions with a bunch of nested where blocks and case expressions. Then you get fewer type signatures, logic that is harder to test, and less reuse.

Also, if we consider a language such as Agda or Idris, splitting proof obligations out to a different function can make development much easier by having fewer things to deal with in your short term memory. Edwin Brady demonstrates this in his talk on Type-driven Development with Idris

Which is really my point. CC is an incredibly simplistic heuristic for something that is inherently difficult to measure, and trying to optimize your code specifically for it will make it worse, rather than better.

While being a simple heuristic as you say, I think it often hits well. In particular, my view is that a lot of Rust code already tends to involve too many long and deep functions (spanning 300 lines) and very long files that make it hard (at least for me) to gauge the overall picture. I think modules, as a consequence, are too underused in Rust. As such, I would much prefer to retain the warn-by-default nature of the cyclomatic_complexity lint in the complexity group.

The programmer needs to consider trade-offs when it comes to readability, for instance the trade-off between rightward-drift and extra indirection. This requires good judgment.

It does. But I find that it also depends on how well the out-extracted function is named. A small and short function calling other aptly named small and short functions can read like pseudo-code. A good IDE can also enable you to quickly jump to the definition of the helper functions. Therefore, I think the default setting should be to write short and sweet definitions.

I'd really prefer if the warn-by-default lints would stick to recommendations that make your code unambiguously better (like 'use a byte literal rather than casting char literals to bytes').

For the reasons I've outlined above, I disagree. To me, lints such as too_many_arguments, type_complexity, and cyclomatic_complexity induce good habits, even if I've been hit by them myself and felt like it was unwarranted.

This comment has been minimized.

@fhartwig

fhartwig Jun 21, 2018

This is sometimes true, but not universally.

Of course. :) Very few things are universally true. In this case I'd say it is more often true than not.

Most suggestions in this lint category actually are always or nearly always improvements. I can't think of any reason to ever write if x == true or x + 0 or if c { true } else {false } or 'c' as u8, while this lint (as well as the two others you mention, which I also don't think should warn by default) may improve the code or make it worse, depending on the situation and personal taste.

I'm not saying this is a bad lint or that high cyclomatic complexity is great, which your reply seems to be arguing against. I'm just saying it doesn't belong together with the other cut-and-dried 'here are clear and simple instructions to make your code better' lints.

@fhartwig

fhartwig Jun 21, 2018

This is sometimes true, but not universally.

Of course. :) Very few things are universally true. In this case I'd say it is more often true than not.

Most suggestions in this lint category actually are always or nearly always improvements. I can't think of any reason to ever write if x == true or x + 0 or if c { true } else {false } or 'c' as u8, while this lint (as well as the two others you mention, which I also don't think should warn by default) may improve the code or make it worse, depending on the situation and personal taste.

I'm not saying this is a bad lint or that high cyclomatic complexity is great, which your reply seems to be arguing against. I'm just saying it doesn't belong together with the other cut-and-dried 'here are clear and simple instructions to make your code better' lints.

This comment has been minimized.

@oli-obk

oli-obk Jun 21, 2018

Contributor

The same reasoning applies to all lints with thresholds. Type complexity and number of single char bindings have arbitrary but sensible thresholds from which on they'll tell you to do "better". In super-mathy code, many single char bindings might be fine. Or you have a very deeply nested type that is obvious due to many different nicely named types.

If you encounter a situation where the threshold does not fit your use case, increase it. If you regress even further, clippy will tell you again with the new threshold. I created the complexity lint group exactly for these kinds of lints, but at some point figured that other things like x == true belonged here, too, because they are also things that are "too complex" for some notion of "too".

I agree that there are two different kinds of lints in the complexity group, but I also don't see how to reasonably split them up further (without just splitting out all the thresholded lints). Categorization always comes with a little "error" because otherwise you'd end up with as many categories as you have stuff to categorize.

@oli-obk

oli-obk Jun 21, 2018

Contributor

The same reasoning applies to all lints with thresholds. Type complexity and number of single char bindings have arbitrary but sensible thresholds from which on they'll tell you to do "better". In super-mathy code, many single char bindings might be fine. Or you have a very deeply nested type that is obvious due to many different nicely named types.

If you encounter a situation where the threshold does not fit your use case, increase it. If you regress even further, clippy will tell you again with the new threshold. I created the complexity lint group exactly for these kinds of lints, but at some point figured that other things like x == true belonged here, too, because they are also things that are "too complex" for some notion of "too".

I agree that there are two different kinds of lints in the complexity group, but I also don't see how to reasonably split them up further (without just splitting out all the thresholded lints). Categorization always comes with a little "error" because otherwise you'd end up with as many categories as you have stuff to categorize.

This comment has been minimized.

@Centril

Centril Jun 22, 2018

Contributor

@fhartwig I agree with @oli-obk, but I would also say that I would expect these lints to be in a group called "complexity". I also think that cyclomatic complexity (at some arbitrary good default) warnings are nearly always (but not 100% of the time) improvements. Or enough of the time at least.

@Centril

Centril Jun 22, 2018

Contributor

@fhartwig I agree with @oli-obk, but I would also say that I would expect these lints to be in a group called "complexity". I also think that cyclomatic complexity (at some arbitrary good default) warnings are nearly always (but not 100% of the time) improvements. Or enough of the time at least.

- [option_unwrap_used](https://rust-lang-nursery.github.io/rust-clippy/master/index.html#option_unwrap_used): Checks for `.unwrap()` calls on `Option`s.
- [assign_ops](https://rust-lang-nursery.github.io/rust-clippy/master/index.html#assign_ops): Checks for compound assignment operations (`+=` and
similar).
- [shadow_unrelated](https://rust-lang-nursery.github.io/rust-clippy/master/index.html#shadow_unrelated): Checks for bindings that shadow other bindings already in

This comment has been minimized.

@Manishearth

Manishearth Jul 30, 2018

Member

How do folks feel about moving this lint to pedantic? It actually is a useful bug-catcher, and it's not quite idiomatic to have unrelated shadows, though marginally so.

@Manishearth

Manishearth Jul 30, 2018

Member

How do folks feel about moving this lint to pedantic? It actually is a useful bug-catcher, and it's not quite idiomatic to have unrelated shadows, though marginally so.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Aug 9, 2018

Member

Lint uplifts from the compiler team are listed in rust-lang/rust#53224, as a separate issue-rfc.

Member

Manishearth commented Aug 9, 2018

Lint uplifts from the compiler team are listed in rust-lang/rust#53224, as a separate issue-rfc.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Aug 9, 2018

Member

@oli-obk so which route do you think we should take here for config.toml stability? Declare it unstable for now, or use @phansch's text?

Member

Manishearth commented Aug 9, 2018

@oli-obk so which route do you think we should take here for config.toml stability? Declare it unstable for now, or use @phansch's text?

@oli-obk

This comment has been minimized.

Show comment
Hide comment
@oli-obk

oli-obk Aug 9, 2018

Contributor

I don't really want to have to support it forever, especially since rls+clippy ignores it already I think. We could make it unstable but make it available via UNSTABLE_CLIPPY_TOML=1 so everyone using it is aware that it might go away after we have a cargo solution. We can have a grace period after the cargo solution is a thing. It could be as long as whatever LTS period is decided on.

Contributor

oli-obk commented Aug 9, 2018

I don't really want to have to support it forever, especially since rls+clippy ignores it already I think. We could make it unstable but make it available via UNSTABLE_CLIPPY_TOML=1 so everyone using it is aware that it might go away after we have a cargo solution. We can have a grace period after the cargo solution is a thing. It could be as long as whatever LTS period is decided on.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Aug 9, 2018

Member

I also think it's fine to have it be declared unstable but not opt in, clippy itself is an opt in tool.

Member

Manishearth commented Aug 9, 2018

I also think it's fine to have it be declared unstable but not opt in, clippy itself is an opt in tool.

@oli-obk

This comment has been minimized.

Show comment
Hide comment
@oli-obk

oli-obk Aug 9, 2018

Contributor

If we don't make it opt-in, doesn't that count as insta-stable?

We could require the clippy.toml file to start with

# I solemly swear that I know clippy.toml is unstable

Kind of the inverse of the MIR human readable output disclaimer

Contributor

oli-obk commented Aug 9, 2018

If we don't make it opt-in, doesn't that count as insta-stable?

We could require the clippy.toml file to start with

# I solemly swear that I know clippy.toml is unstable

Kind of the inverse of the MIR human readable output disclaimer

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Aug 31, 2018

Member

Added 9b7ab4c which lists the proposed changes. Basically:

  • shadow_unrelated is promoted to a pedantic lint
  • We have a bunch of renames and uplifts being tracked in rust-lang/rust#53224

AFAICT these are the only proposed changes that seemed to have some consensus -- there was one other, to make cyclomatic_complexity Allow but it didn't seem to have consensus. Let me know if you think I've missed something.

I kept the changes as a separate list instead of working it into the main lists because it's easier to deal with that way if the proposals change.

Member

Manishearth commented Aug 31, 2018

Added 9b7ab4c which lists the proposed changes. Basically:

  • shadow_unrelated is promoted to a pedantic lint
  • We have a bunch of renames and uplifts being tracked in rust-lang/rust#53224

AFAICT these are the only proposed changes that seemed to have some consensus -- there was one other, to make cyclomatic_complexity Allow but it didn't seem to have consensus. Let me know if you think I've missed something.

I kept the changes as a separate list instead of working it into the main lists because it's easier to deal with that way if the proposals change.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Aug 31, 2018

Member

@rfcbot fcp merge

We discussed this in Tuesday's meeting, seems like the dust has settled and it's ready to merge.

Member

Manishearth commented Aug 31, 2018

@rfcbot fcp merge

We discussed this in Tuesday's meeting, seems like the dust has settled and it's ready to merge.

@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Aug 31, 2018

Team member @Manishearth has proposed to merge this. The next step is review by the rest of the tagged teams:

Concerns:

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

rfcbot commented Aug 31, 2018

Team member @Manishearth has proposed to merge this. The next step is review by the rest of the tagged teams:

Concerns:

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@killercup

This comment has been minimized.

Show comment
Hide comment
@killercup

killercup Aug 31, 2018

Member

@rfcbot reviewed

Let's go for it!

Member

killercup commented Aug 31, 2018

@rfcbot reviewed

Let's go for it!

@joshtriplett

This comment has been minimized.

Show comment
Hide comment
@joshtriplett

joshtriplett Sep 2, 2018

Member

I'm not on T-dev-tools, but speaking personally and with a T-lang hat on, I'd like to request one lint change here:

Could we please move explicit_iter_loop to "Allow" rather than "Warn"?

The justification for that lint just says "Readability", but often, for x in something.iter() can make a loop more readable. And in any case, this isn't universally idiomatic Rust style. This is a particular mental model of references, in which &something isn't "take a reference to something" but rather "switch into reference mode".

I'm not arguing that the lint shouldn't exist, but I do think it's something that people should be able to optionally turn on in codebases that want to use that style, rather than something warned on by default.

cc @aturon

See also rust-lang-nursery/rust-clippy#3063

Member

joshtriplett commented Sep 2, 2018

I'm not on T-dev-tools, but speaking personally and with a T-lang hat on, I'd like to request one lint change here:

Could we please move explicit_iter_loop to "Allow" rather than "Warn"?

The justification for that lint just says "Readability", but often, for x in something.iter() can make a loop more readable. And in any case, this isn't universally idiomatic Rust style. This is a particular mental model of references, in which &something isn't "take a reference to something" but rather "switch into reference mode".

I'm not arguing that the lint shouldn't exist, but I do think it's something that people should be able to optionally turn on in codebases that want to use that style, rather than something warned on by default.

cc @aturon

See also rust-lang-nursery/rust-clippy#3063

@joshtriplett

This comment has been minimized.

Show comment
Hide comment
@joshtriplett

joshtriplett Sep 2, 2018

Member

One question for clarification: post-1.0, will it still be possible to decrease the severity of lints ("Deny" -> "Warn" or "Warn" -> "Allow")?

Member

joshtriplett commented Sep 2, 2018

One question for clarification: post-1.0, will it still be possible to decrease the severity of lints ("Deny" -> "Warn" or "Warn" -> "Allow")?

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Sep 2, 2018

Member

I'll proxy-concern that. It hasn't been brought up so far.

@rfcbot concern explicit-iter-loop-level

See #2476 (comment)

Member

Manishearth commented Sep 2, 2018

I'll proxy-concern that. It hasn't been brought up so far.

@rfcbot concern explicit-iter-loop-level

See #2476 (comment)

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Sep 2, 2018

Member

Personally I feel that &foo has been more idiomatic for a while, but that's anecdotal. We could move it to pedantic. So personally I'd like to see it stay where it is but it's a super weak preference and you've put forth decent reasons.

Member

Manishearth commented Sep 2, 2018

Personally I feel that &foo has been more idiomatic for a while, but that's anecdotal. We could move it to pedantic. So personally I'd like to see it stay where it is but it's a super weak preference and you've put forth decent reasons.

@joshtriplett

This comment has been minimized.

Show comment
Hide comment
@joshtriplett

joshtriplett Sep 2, 2018

Member

@Manishearth Thank you, I appreciate it.

Member

joshtriplett commented Sep 2, 2018

@Manishearth Thank you, I appreciate it.

@dwijnand

This comment has been minimized.

Show comment
Hide comment
@dwijnand

dwijnand Sep 2, 2018

Member

The justification for that lint just says "Readability", but often, for x in something.iter() can make a loop more readable.

Perhaps explicit_into_iter_loop too.

By the way, from GitHub search it looks like lots of people opt-out of explicit_iter_loop: https://github.com/search?p=2&q=allow%28explicit_iter_loop%29&type=Code

Member

dwijnand commented Sep 2, 2018

The justification for that lint just says "Readability", but often, for x in something.iter() can make a loop more readable.

Perhaps explicit_into_iter_loop too.

By the way, from GitHub search it looks like lots of people opt-out of explicit_iter_loop: https://github.com/search?p=2&q=allow%28explicit_iter_loop%29&type=Code

@joshtriplett

This comment has been minimized.

Show comment
Hide comment
@joshtriplett

joshtriplett Sep 2, 2018

Member

@dwijnand Including Cargo, it seems: https://github.com/rust-lang/cargo/blob/master/src/cargo/lib.rs

I feel a little less strongly about explicit_into_iter_loop, but I'd certainly not complain if it ended up switched to "Allow" as well.

Member

joshtriplett commented Sep 2, 2018

@dwijnand Including Cargo, it seems: https://github.com/rust-lang/cargo/blob/master/src/cargo/lib.rs

I feel a little less strongly about explicit_into_iter_loop, but I'd certainly not complain if it ended up switched to "Allow" as well.

@joshtriplett

This comment has been minimized.

Show comment
Hide comment
@joshtriplett

joshtriplett Sep 3, 2018

Member

@Manishearth Pull request, to go along with the requested lint reclassification: rust-lang-nursery/rust-clippy#3119 😁

Member

joshtriplett commented Sep 3, 2018

@Manishearth Pull request, to go along with the requested lint reclassification: rust-lang-nursery/rust-clippy#3119 😁

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Sep 3, 2018

Member

It merged and there seemed to be general ageement

@rfcbot resolve explicit-iter-loop-level

Member

Manishearth commented Sep 3, 2018

It merged and there seemed to be general ageement

@rfcbot resolve explicit-iter-loop-level

@phansch

This comment has been minimized.

Show comment
Hide comment
@phansch

phansch Sep 5, 2018

AFAICT one open question is regarding the clippy.toml.

I think what @oli-obk said can be a good solution.
We could require a new config to be set that's called clippy_toml_is_unstable_and_may_go_away.
If it is not set, it will show a warning when Clippy is executed.
Once the config is included, the warning will go away.

How would this work with the RLS <-> Clippy integration, though?

phansch commented Sep 5, 2018

AFAICT one open question is regarding the clippy.toml.

I think what @oli-obk said can be a good solution.
We could require a new config to be set that's called clippy_toml_is_unstable_and_may_go_away.
If it is not set, it will show a warning when Clippy is executed.
Once the config is included, the warning will go away.

How would this work with the RLS <-> Clippy integration, though?

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Sep 5, 2018

Member

@rfcbot concern clippy-toml-stability

Should we be stabilizing it? Or declare it unstable for now a la #2476 (comment) ?

Member

Manishearth commented Sep 5, 2018

@rfcbot concern clippy-toml-stability

Should we be stabilizing it? Or declare it unstable for now a la #2476 (comment) ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment