Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Applied clippy lints #1734

Merged
merged 6 commits into from
Feb 23, 2023
Merged

Applied clippy lints #1734

merged 6 commits into from
Feb 23, 2023

Conversation

Bravo555
Copy link
Contributor

@Bravo555 Bravo555 commented Feb 17, 2023

Proposed changes

Using rust 1.69, there were quite a lot of lints reported by cargo clippy. Most of them are applied in this commit, however one particular type of lint remains and there were some ambiguities.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
  • Documentation Update (if none of the other choices apply)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue


Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA (in all commits with git commit -s)
  • I ran cargo fmt as mentioned in CODING_GUIDELINES
  • I used cargo clippy as mentioned in CODING_GUIDELINES
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

Most of the lints were very boring things, like removing unnecessary borrows.

One type of lint I did not correct, which is the Err-variant returned from this function is very large lint which lints large error types returned in Results because result is as large as its largest variant, and in some cases I've seen function that have Ok(()) variant with Err variants being in excess of 200 bytes, which might not be optimal, but fixing it would involve boxing stuff and changing behaviour, so this should be looked at in the separate PR.

Finally, there was one ambiguity i encountered:

Err(err) => {
let msg = format!("Error: {:?}", err);
let _ = sender.send(msg).await;
break;
}

This was changed to await the future as I figured this must've been the intent.

Finally, when merging the PR, can we use squash merge for the PR? I've divided it into commits for review purposes, but it can be merged as 1 commit, and Github can do that one itself.

@Bravo555 Bravo555 temporarily deployed to Test Pull Request February 17, 2023 18:52 — with GitHub Actions Inactive
@albinsuresh
Copy link
Contributor

One type of lint I did not correct, which is the Err-variant returned from this function is very large lint which lints large error types returned in Results because result is as large as its largest variant, and in some cases I've seen function that have Ok(()) variant with Err variants being in excess of 200 bytes, which might not be optimal, but fixing it would involve boxing stuff and changing behaviour, so this should be looked at in the separate PR.

Ignore that lint for now.

Finally, when merging the PR, can we use squash merge for the PR? I've divided it into commits for review purposes, but it can be merged as 1 commit, and Github can do that one itself.

Yes, we've enabled the squash merge option precisely for that reason.

@didier-wenzek
Copy link
Contributor

One type of lint I did not correct, which is the Err-variant returned from this function is very large lint which lints large error types returned in Results because result is as large as its largest variant, and in some cases I've seen function that have Ok(()) variant with Err variants being in excess of 200 bytes, which might not be optimal, but fixing it would involve boxing stuff and changing behaviour, so this should be looked at in the separate PR.

Yes, this has to be done in a specific PR. See #1716

@github-actions
Copy link
Contributor

github-actions bot commented Feb 20, 2023

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass %
137 0 5 137 100

Passed Tests

Name ⏱️ Duration Suite
Define Child device 1 ID 0.009 s C8Y Child Alarms Rpi
Normal case when the child device does not exist on c8y cloud 2.5620000000000003 s C8Y Child Alarms Rpi
Normal case when the child device already exists 0.834 s C8Y Child Alarms Rpi
Reconciliation when the new alarm message arrives, restart the mapper 2.219 s C8Y Child Alarms Rpi
Reconciliation when the alarm that is cleared 5.663 s C8Y Child Alarms Rpi
Prerequisite Parent 19.786 s Child Conf Mgmt Plugin
Prerequisite Child 0.41 s Child Conf Mgmt Plugin
Child device bootstrapping 16.691 s Child Conf Mgmt Plugin
Snapshot from device 19.806 s Child Conf Mgmt Plugin
Child device config update 17.361 s Child Conf Mgmt Plugin
Configuration types should be detected on file change (without restarting service) 50.116 s Inotify Crate
Child devices support sending simple measurements 48.234 s Child Device Telemetry
Child devices support sending custom measurements 48.307 s Child Device Telemetry
Child devices support sending custom events 41.11 s Child Device Telemetry
Child devices support sending custom events overriding the type 36.483 s Child Device Telemetry
Child devices support sending custom alarms #1699 35.813 s Child Device Telemetry
Child devices support sending inventory data via c8y topic 23.204 s Child Device Telemetry
Main device support sending inventory data via c8y topic 26.312 s Child Device Telemetry
Successful firmware operation 67.554 s Firmware Operation
Install with empty firmware name 50.984 s Firmware Operation
Supports restarting the device 81.811 s Restart Device
Update tedge version from previous using Cumulocity 111.437 s Tedge Self Update
Successful shell command with output 4.165 s Shell Operation
Check Successful shell command with literal double quotes output 3.429 s Shell Operation
Execute multiline shell command 3.338 s Shell Operation
Failed shell command 3.443 s Shell Operation
Software list should be populated during startup 53.6 s Software
Install software via Cumulocity 67.634 s Software
Software list should only show currently installed software and not candidates 47.379 s Software
Stop tedge-agent service 0.298 s Log Path Config
Customize the log path 0.151 s Log Path Config
Initialize tedge-agent 0.159 s Log Path Config
Check created folders 0.087 s Log Path Config
Remove created custom folders 0.132 s Log Path Config
Install latest via script (from current branch) 31.722 s Install Tedge
Install specific version via script (from current branch) 22.191 s Install Tedge
Install latest tedge via script (from main branch) 21.263 s Install Tedge
Support starting and stopping services 38.979 s Service-Control
Supports a reconnect 45.688 s Test-Commands
Supports disconnect then connect 49.545 s Test-Commands
Update unknown setting 31.744 s Test-Commands
Update known setting 26.218 s Test-Commands
Stop c8y-configuration-plugin 0.124 s Health C8Y-Configuration-Plugin
Update the service file 0.122 s Health C8Y-Configuration-Plugin
Reload systemd files 0.477 s Health C8Y-Configuration-Plugin
Start c8y-configuration-plugin 0.19 s Health C8Y-Configuration-Plugin
Start watchdog service 10.228 s Health C8Y-Configuration-Plugin
Check PID of c8y-configuration-plugin 0.272 s Health C8Y-Configuration-Plugin
Kill the PID 0.147 s Health C8Y-Configuration-Plugin
Recheck PID of c8y-configuration-plugin 2.27 s Health C8Y-Configuration-Plugin
Compare PID change 0.001 s Health C8Y-Configuration-Plugin
Stop watchdog service 0.148 s Health C8Y-Configuration-Plugin
Remove entry from service file 0.167 s Health C8Y-Configuration-Plugin
Stop c8y-log-plugin 0.258 s Health C8Y-Log-Plugin
Update the service file 0.156 s Health C8Y-Log-Plugin
Reload systemd files 1.259 s Health C8Y-Log-Plugin
Start c8y-log-plugin 0.373 s Health C8Y-Log-Plugin
Start watchdog service 10.345 s Health C8Y-Log-Plugin
Check PID of c8y-log-plugin 0.12 s Health C8Y-Log-Plugin
Kill the PID 0.152 s Health C8Y-Log-Plugin
Recheck PID of c8y-log-plugin 2.18 s Health C8Y-Log-Plugin
Compare PID change 0.001 s Health C8Y-Log-Plugin
Stop watchdog service 0.112 s Health C8Y-Log-Plugin
Remove entry from service file 0.085 s Health C8Y-Log-Plugin
Stop tedge-mapper 0.323 s Health Tedge Mapper C8Y
Update the service file 0.25 s Health Tedge Mapper C8Y
Reload systemd files 1.223 s Health Tedge Mapper C8Y
Start tedge-mapper 0.294 s Health Tedge Mapper C8Y
Start watchdog service 10.312 s Health Tedge Mapper C8Y
Check PID of tedge-mapper 0.078 s Health Tedge Mapper C8Y
Kill the PID 0.075 s Health Tedge Mapper C8Y
Recheck PID of tedge-mapper 2.176 s Health Tedge Mapper C8Y
Compare PID change 0.001 s Health Tedge Mapper C8Y
Stop watchdog service 0.118 s Health Tedge Mapper C8Y
Remove entry from service file 0.081 s Health Tedge Mapper C8Y
Stop tedge-agent 0.139 s Health Tedge-Agent
Update the service file 0.073 s Health Tedge-Agent
Reload systemd files 0.411 s Health Tedge-Agent
Start tedge-agent 0.101 s Health Tedge-Agent
Start watchdog service 10.137 s Health Tedge-Agent
Check PID of tedge-mapper 0.073 s Health Tedge-Agent
Kill the PID 0.069 s Health Tedge-Agent
Recheck PID of tedge-agent 2.19 s Health Tedge-Agent
Compare PID change 0.005 s Health Tedge-Agent
Stop watchdog service 0.131 s Health Tedge-Agent
Remove entry from service file 0.125 s Health Tedge-Agent
Stop tedge-mapper-az 0.23 s Health Tedge-Mapper-Az
Update the service file 0.214 s Health Tedge-Mapper-Az
Reload systemd files 0.833 s Health Tedge-Mapper-Az
Start tedge-mapper-az 0.266 s Health Tedge-Mapper-Az
Start watchdog service 10.352 s Health Tedge-Mapper-Az
Check PID of tedge-mapper-az 0.322 s Health Tedge-Mapper-Az
Kill the PID 0.372 s Health Tedge-Mapper-Az
Recheck PID of tedge-agent 2.2560000000000002 s Health Tedge-Mapper-Az
Compare PID change 0.01 s Health Tedge-Mapper-Az
Stop watchdog service 0.285 s Health Tedge-Mapper-Az
Remove entry from service file 0.313 s Health Tedge-Mapper-Az
Stop tedge-mapper-collectd 0.269 s Health Tedge-Mapper-Collectd
Update the service file 0.163 s Health Tedge-Mapper-Collectd
Reload systemd files 0.856 s Health Tedge-Mapper-Collectd
Start tedge-mapper-collectd 0.264 s Health Tedge-Mapper-Collectd
Start watchdog service 10.453 s Health Tedge-Mapper-Collectd
Check PID of tedge-mapper-collectd 0.179 s Health Tedge-Mapper-Collectd
Kill the PID 0.28 s Health Tedge-Mapper-Collectd
Recheck PID of tedge-mapper-collectd 2.471 s Health Tedge-Mapper-Collectd
Compare PID change 0.001 s Health Tedge-Mapper-Collectd
Stop watchdog service 0.335 s Health Tedge-Mapper-Collectd
Remove entry from service file 0.129 s Health Tedge-Mapper-Collectd
c8y-log-plugin health status 5.824 s MQTT health endpoints
c8y-configuration-plugin health status 6.134 s MQTT health endpoints
Wrong package name 0.229 s Improve Tedge Apt Plugin Error Messages
Wrong version 0.348 s Improve Tedge Apt Plugin Error Messages
Wrong type 0.935 s Improve Tedge Apt Plugin Error Messages
tedge_connect_test_positive 0.549 s Tedge Connect Test
tedge_connect_test_negative 1.102 s Tedge Connect Test
tedge_connect_test_sm_services 7.258 s Tedge Connect Test
tedge_disconnect_test_sm_services 0.359 s Tedge Connect Test
Install thin-edge.io 14.112 s Call Tedge
call tedge -V 0.286 s Call Tedge
call tedge -h 0.212 s Call Tedge
call tedge -h -V 0.288 s Call Tedge
call tedge help 0.231 s Call Tedge
tedge config list 0.088 s Call Tedge Config List
tedge config list --all 0.088 s Call Tedge Config List
set/unset device.type 0.487 s Call Tedge Config List
set/unset device.key.path 0.532 s Call Tedge Config List
set/unset device.cert.path 0.523 s Call Tedge Config List
set/unset c8y.root.cert.path 0.502 s Call Tedge Config List
set/unset c8y.smartrest.templates 0.517 s Call Tedge Config List
set/unset az.root.cert.path 0.614 s Call Tedge Config List
set/unset az.mapper.timestamp 0.541 s Call Tedge Config List
set/unset mqtt.bind_address 0.282 s Call Tedge Config List
set/unset mqtt.port 0.353 s Call Tedge Config List
set/unset tmp.path 0.378 s Call Tedge Config List
set/unset logs.path 0.419 s Call Tedge Config List
set/unset run.path 0.415 s Call Tedge Config List
Get Put Delete 3.928 s Http File Transfer Api

Copy link
Contributor

@didier-wenzek didier-wenzek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would approve these changed with the exception of the as u64 fix. The later makes sense but we need to double check as this has already led to an issue #592.

crates/common/download/src/download.rs Outdated Show resolved Hide resolved
@didier-wenzek
Copy link
Contributor

Finally, when merging the PR, can we use squash merge for the PR? I've divided it into commits for review purposes, but it can be merged as 1 commit, and Github can do that one itself.

We avoid squash merge. Here I would make an interactive rebase to separate the boring fixes from those subtle enough to have deserved some comments.

Copy link
Contributor

@albinsuresh albinsuresh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. But, I'm still wondering why these clippy warnings were never emitted earlier, even with the clippy checks that are part of our CI.

crates/core/tedge_api/src/group.rs Show resolved Hide resolved
@Bravo555 Bravo555 marked this pull request as draft February 21, 2023 12:08
@Bravo555 Bravo555 marked this pull request as ready for review February 21, 2023 12:40
@Bravo555 Bravo555 marked this pull request as draft February 21, 2023 12:40
@Bravo555 Bravo555 temporarily deployed to Test Pull Request February 21, 2023 12:47 — with GitHub Actions Inactive
@Bravo555
Copy link
Contributor Author

Bravo555 commented Feb 21, 2023

Okay, so the reason these lints were not caught either on CI or on local dev environments, is because rust toolchain version is pinned to 1.64 on CI and on dev environments (via rust-toolchain file) so when a bunch of lints got added in 1.65, 1.66, 1.67, etc. they did not appear.

But at the same time, we use imports_granularity, which is still unstable and requires using a nightly version of the compiler.
So what I think often happens, people use the stable toolchain and either don't change the imports usually, or when they do, they use cargo fmt when CI doesn't pass, and don't bother to run clippy manually. Then on CI clippy is ran on version 1.64 and new lints introduced in new rust versions are ignored. As time goes on, more lints get added and we fall behind.

So on the one hand, we check lints with 1.64 because it's our MSRV, and on the other we use features that are in nightly, while still keeping rust-toolchain that overrides the toolchain version to 1.64 on local dev environments.

But clippy knows about the MSRV and some lints are gated behind MSRV so that lints that take advantage of newer features are not shown when below MSRV specified in the manifest. So there is no reason to use rust-toolchain file for clippy.

So I propose the following:

  • Remove the rust-toolchain file, do local development or nightly as long as import_granularity is not stabilised, then if it is, people can either change to stable or keep using nightly: This can increase the CI build failures as people can locally use new features which are not present on 1.64 build CI, but then we're aware and can either defer using new features or bump MSRV.
  • Run the builds on MSRV (1.64): In some environments like yocto, or other reasons, a user might want to build the project but is not able to use the latest compiler. In such cases we check if we can build on MSRV, and consider bumping it in case we use some new feature.
  • Don't constrain rust version for tests, they can be run on stable or nightly: Tests are used for development purposes, locally or on CI. Developers can be expected to use latest version of Rust, and end users usually don't run them, so we can use new features in tests.
  • Finally, use stable or nightly clippy: If using nightly, we get latest lints before they appear on stable and have more time to apply them, if any issues arise out of that, we can reconsider switching to stable.

I first stumbled upon this when revising MSRV for Yocto, and thought that it's not relevant, so I didn't mention it, yet it came up almost immediately. So I think now I'll be reporting this kind of stuff immediately as it comes up.

@didier-wenzek @PradeepKiruvale Let me know what you think.

@didier-wenzek
Copy link
Contributor

So on the one hand, we check lints with 1.64 because it's our MSRV, and on the other we use features that are in nightly, while still keeping rust-toolchain that overrides the toolchain version to 1.64 on local dev environments.

Not fully correct. We are using nightly *only for cargo fmt.

Remove the rust-toolchain file

I'm okay with that. This is indeed a bit confusing when trying to update locally for a more recent toolchain.

Run the builds on MSRV (1.64)

Agree to change nothing here.

Don't constrain rust version for tests, they can be run on stable or nightly

Okay on each developer's fork/clone but why also on the CI?

use stable or nightly clippy

I would encourage the developers to use nightly clippy on their fork/clone but would keep stable for CI. So, as you said, we can address clippy lint issues sooner, but without being forced.

@Bravo555
Copy link
Contributor Author

Bravo555 commented Feb 21, 2023

Don't constrain rust version for tests, they can be run on stable or nightly

Okay on each developer's fork/clone but why also on the CI?

It's so that we can use new features in tests without them failing on the CI, because tests are only run during development and not by the end users, who only care about building the binary. I admit that there's not much room to utilise new features in unit and integration tests, but in theory there's nothing that would prohibit as from doing so, so we might as well do it - was my thinking.

@PradeepKiruvale
Copy link
Contributor

So on the one hand, we check lints with 1.64 because it's our MSRV, and on the other we use features that are in nightly, while still keeping rust-toolchain that overrides the toolchain version to 1.64 on local dev environments.

Not fully correct. We are using nightly *only for cargo fmt.

Remove the rust-toolchain file

I'm okay with that. This is indeed a bit confusing when trying to update locally for a more recent toolchain.

Run the builds on MSRV (1.64)

Agree to change nothing here.

Don't constrain rust version for tests, they can be run on stable or nightly

Okay on each developer's fork/clone but why also on the CI?

use stable or nightly clippy

I would encourage the developers to use nightly clippy on their fork/clone but would keep stable for CI. So, as you said, we can address clippy lint issues sooner, but without being forced.

I am also ok with using nightly clippy. But there might be a lot of warnings to be fixed now.

Copy link
Contributor

@didier-wenzek didier-wenzek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@@ -222,7 +223,7 @@ fn create_file_and_try_pre_allocate_space(
file.as_raw_fd(),
FallocateFlags::empty(),
0,
file_len as nix::libc::off_t,
file_len.try_into().expect("file too large to fit in i64"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expect will panic right? @didier-wenzek Is it ok to have that here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice it will never panic, because for the u64 -> i64 conversion to fail, the file would have to be bigger than 2^63-1, which comes up to roughly 8 exabytes. Maybe it is a meaningless change, but file_len is a function parameter, and if somebody manages to call it so horribly wrong, this would panic whereas as would've silently proceeded, by either truncating or overflowing, not sure which. Not that it matters because the called function would also probably return an error, but in principle, using .into()/.try_into() is preferred to as, so why not. Will probably look into converting as to using into in the rest of the codebase as well some other time.

@@ -174,7 +172,8 @@ mod tests {

#[test]
fn control_chars_are_removed() {
let input = generate_test_vec_u8();
let input: Vec<u8> = (0x00..0xff).collect();
dbg!(&input);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove dbg!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh my bad, was testing if it will really be the same and forgot to remove it.

"{\"type\":\"ThinEdgeMeasurement\",\"temperature0\":{\"temperature0\":{\"value\":0.0}}"
));
assert!(result[0].payload_str().unwrap().contains(
"{\"type\":\"ThinEdgeMeasurement\",\"temperature0\":{\"temperature0\":{\"value\":0.0}}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not raw string here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to convert, will fix

Copy link
Contributor

@PradeepKiruvale PradeepKiruvale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left a few comments.

Signed-off-by: Marcel Guzik <marcel.guzik@inetum.com>
Signed-off-by: Marcel Guzik <marcel.guzik@inetum.com>
Signed-off-by: Marcel Guzik <marcel.guzik@inetum.com>
Signed-off-by: Marcel Guzik <marcel.guzik@inetum.com>
@Bravo555 Bravo555 marked this pull request as ready for review February 21, 2023 16:06
Signed-off-by: Marcel Guzik <marcel.guzik@inetum.com>
Signed-off-by: Marcel Guzik <marcel.guzik@inetum.com>
@Bravo555 Bravo555 temporarily deployed to Test Pull Request February 21, 2023 16:55 — with GitHub Actions Inactive
@didier-wenzek
Copy link
Contributor

didier-wenzek commented Feb 21, 2023

This test is flaky:

---- c8y::tests::c8y_mapper_alarm_mapping_to_smartrest stdout ----
thread 'c8y::tests::c8y_mapper_alarm_mapping_to_smartrest' panicked at 'Didn't receive all expected messages: ["302,temperature_alarm,\"Temperature high\""]
 Received: ["500\n", "105"]', /home/runner/work/thin-edge.io/thin-edge.io/crates/tests/mqtt_tests/src/test_mqtt_client.rs:89:5

Created an issue: #1741

@Bravo555
Copy link
Contributor Author

This test is flaky:

Will fix.

One more thing: clippy has even more lints groups that are allowed by default. I tried changing clippy::pedantic to warn, and there was an absolute monstrous number of problems, don't know exactly how many because cargo reports per package rather than a workspace total and I'm not sure how to change that, but there were easily more than 200, with cargo clippy --fix touching another 100 files.

You can run cargo clippy -- -Aclippy::result-large-err -Wclippy::pedantic and decide whether or not we do we bother with these lints or is this enough for one PR 😅

@didier-wenzek
Copy link
Contributor

You can run cargo clippy -- -Aclippy::result-large-err -Wclippy::pedantic and decide whether or not we do we bother with these lints or is this enough for one PR

I would not bother too much with pedantic warnings. Indeed, numerous but not really useful as "consider using an underscore-prefixed named binding" instead of _ . or "item in documentation is missing backticks".

@Bravo555
Copy link
Contributor Author

This test is flaky:

Will fix.

Never mind, I did not notice this issue was assigned to you @PradeepKiruvale, I'll leave the fix to you then.

@reubenmiller reubenmiller merged commit 28ba0f6 into thin-edge:main Feb 23, 2023
@Bravo555 Bravo555 deleted the clippy branch February 23, 2023 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants