Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix recent clippy errors on master #941

Merged
merged 3 commits into from
Aug 6, 2021
Merged

Conversation

thanethomson
Copy link
Member

@thanethomson thanethomson commented Aug 5, 2021

Some new clippy errors have appeared recently on master (most likely due to upgrading to Rust 1.54). This PR aims to fix them.

See https://github.com/informalsystems/tendermint-rs/runs/3242281271

  • Referenced an issue explaining the need for the change
  • Updated all relevant documentation in docs
  • Updated all code comments where relevant
  • Wrote tests
  • Added entry in .changelog/

Signed-off-by: Thane Thomson <connect@thanethomson.com>
Signed-off-by: Thane Thomson <connect@thanethomson.com>
@thanethomson thanethomson requested a review from xla August 5, 2021 11:35
Signed-off-by: Thane Thomson <connect@thanethomson.com>
@codecov-commenter
Copy link

Codecov Report

Merging #941 (a606c1d) into master (09c8454) will increase coverage by 0.1%.
The diff coverage is 92.3%.

Impacted file tree graph

@@           Coverage Diff            @@
##           master    #941     +/-   ##
========================================
+ Coverage    70.5%   70.7%   +0.1%     
========================================
  Files         205     205             
  Lines       16470   16433     -37     
========================================
- Hits        11626   11622      -4     
+ Misses       4844    4811     -33     
Impacted Files Coverage Δ
light-client/src/evidence.rs 50.0% <ø> (ø)
light-client/src/macros.rs 10.3% <ø> (ø)
light-client/src/store/memory.rs 66.6% <ø> (ø)
tendermint/src/chain/id.rs 73.8% <0.0%> (ø)
light-client/src/components/verifier.rs 100.0% <100.0%> (ø)
light-client/src/peer_list.rs 90.3% <100.0%> (ø)
light-client/src/predicates.rs 100.0% <100.0%> (ø)
light-client/src/supervisor.rs 90.6% <100.0%> (ø)
tendermint/src/abci/tag.rs 74.0% <100.0%> (ø)
testgen/src/tester.rs 72.2% <100.0%> (ø)
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 09c8454...a606c1d. Read the comment docs.

Copy link
Member

@romac romac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question, pre-approving already otherwise

@@ -81,6 +81,7 @@ impl LightStore for MemoryStore {
.map(|(_, e)| e.light_block.clone())
}

#[allow(clippy::needless_collect)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we just return the boxes iterator directly here? I assume you've tried that already, so just curious what's the issue here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it's a good question. I refactored the code to the following:

    fn all(&self, status: Status) -> Box<dyn Iterator<Item = LightBlock>> {
        let light_blocks = self
            .store
            .iter()
            .filter(|(_, e)| e.status == status)
            .map(|(_, e)| e.light_block.clone());

        Box::new(light_blocks)
    }

Is that what you mean?

If so, this is the error I get from the compiler:

error[E0759]: `self` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
  --> light-client/src/store/memory.rs:87:14
   |
84 |       fn all(&self, status: Status) -> Box<dyn Iterator<Item = LightBlock>> {
   |              ----- this data with an anonymous lifetime `'_`...
85 |           let light_blocks = self
   |  ____________________________-
86 | |             .store
   | |__________________- ...is captured here...
87 |               .iter()
   |                ^^^^
...
91 |           Box::new(light_blocks)
   |           ---------------------- ...and is required to live as long as `'static` here
   |
help: to declare that the trait object captures data from argument `self`, you can add an explicit `'_` lifetime bound
   |
84 |     fn all(&self, status: Status) -> Box<dyn Iterator<Item = LightBlock> + '_> {
   |                                                                          ^^^^

error: aborting due to previous error

I'm not totally comfortable just yet with the way that these lifetime bounds work (due to lack of experience with using them). Does declaring that the trait object captures data from argument 'self' make sense to you? What does this mean, practically?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if I apply the suggestion, we'll need to modify the trait signature:

error: `impl` item signature doesn't match `trait` item signature
  --> light-client/src/store/memory.rs:84:5
   |
84 |     fn all(&self, status: Status) -> Box<dyn Iterator<Item = LightBlock> + '_> {
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ found `fn(&MemoryStore, types::Status) -> Box<dyn Iterator<Item = types::LightBlock>>`
   | 
  ::: light-client/src/store.rs:48:5
   |
48 |     fn all(&self, status: Status) -> Box<dyn Iterator<Item = LightBlock>>;
   |     ---------------------------------------------------------------------- expected `fn(&MemoryStore, types::Status) -> Box<(dyn Iterator<Item = types::LightBlock> + 'static)>`
   |
   = note: expected `fn(&MemoryStore, types::Status) -> Box<(dyn Iterator<Item = types::LightBlock> + 'static)>`
              found `fn(&MemoryStore, types::Status) -> Box<dyn Iterator<Item = types::LightBlock>>`
   = help: the lifetime requirements from the `impl` do not correspond to the requirements in the `trait`
   = help: verify the lifetime relationships in the `trait` and `impl` between the `self` argument, the other inputs and its output

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thanethomson heh, that's a really "fun" syntax it's nearly impossible to discover without assistance from rustc...

That said, why not write it with impl Trait instead of Box + dyn?

    fn all(&self, status: Status) -> impl Iterator<Item = LightBlock> + '_ {
       self
            .store
            .iter()
            .filter(|(_, e)| e.status == status)
            .map(|(_, e)| e.light_block.clone())
    }

Copy link
Member

@romac romac Aug 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tony-iqlusion Because all is a trait method and impl Trait return types in trait methods are not supported (yet?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like closures borrow the variables they close over by default, even if they are Copy. Moving it into the closure will just copy it into it and should just work(tm)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should just return a Vec here rather than an iterator. This code is very suboptimal already (and would deserve a rewrite) so I don't think we mind the extra allocations.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should just return a Vec here rather than an iterator.

That probably makes sense, given that we're cloning light blocks anyways. So should we perhaps look at changing the method signature to:

    /// Get an iterator of all light blocks with the given status.
    fn all(&self, status: Status) -> Vec<LightBlock>;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now though, perhaps we should just leave the #[allow(clippy::needless_collect)]? We can fix this in a follow-up PR, as it touches various aspects of the codebase and is a breaking API change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've opened #943 to follow up on this.

@thanethomson thanethomson merged commit e824bfc into master Aug 6, 2021
@thanethomson thanethomson deleted the thane/fix-clippy-errors branch August 6, 2021 18:26
thanethomson added a commit that referenced this pull request Aug 7, 2021
* Implement full-duplex secret connection (#938)

* Implement thread-safe cloning of a secret connection

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Expand documentation for SecretConnection on threading considerations

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Extract peer construction into its own method

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add test for cloned SecretConnection

This adds a `TcpStream`-based test for parallelizing operations on a
`SecretConnection`. I used `TcpStream` instead of the buffered reader in
the other tests because it wasn't feasible to implement the `TryClone`
trait for that buffered pipe implementation.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add more messages to test

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Expand comment for clarity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add .changelog entry

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Restore half-duplex operations

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Extract encrypt/decrypt fns as independent methods

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove unnecessary trait bounds

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Extract send/receive state

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Extract read/write functionality as standalone methods

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add logic to facilitate splitting SecretConnection into its sending and receiving halves

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Restore split SecretConnection test using new semantics

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update changelog entry

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update docs for `SecretConnection`

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Condense error reporting

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Extract TryClone trait into its own crate

As per the discussion at
#938 (comment),
this extracts the `TryClone` trait into a new crate called
`tendermint-std-ext` in the `std-ext` directory.

This new crate is intended to contain any code that we need that extends
the Rust standard library.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Reorder imports

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Assert validation regardless of debug build

This introduces the internal encryption assertions at runtime regardless
of build type. This may introduce a small performance hit, but it's
probably worth it to ensure correctness.

Effectively this is keeping an eye on the code in the
`encrypt_and_write` fn to ensure its correctness.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove remote_pubkey optionality from sender/receiver halves

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update SecretConnection docs with comment content

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix doc link to TryClone trait

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix doc link to TryClone trait

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add docs on SecretConnection failures and connection integrity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Synchronize sending/receiving failures to comply with crypto algorithm constraints

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Rename try_split method to split for SecretConnection

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove redundant field name prefixes

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix broken link in docs

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix recent clippy errors on `master` (#941)

* Fix needless borrows in codebase

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Ignore needless collect warning (we do actually seem to need it)

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove trailing semicolon in macro to fix docs compiling

Signed-off-by: Thane Thomson <connect@thanethomson.com>
thanethomson added a commit that referenced this pull request Aug 7, 2021
* Use flex-error for tendermint

* Use flex-error for p2p

* Use flex-error for light-client

* Use flex-error for light_client::predicates

* Fix lint

* Use flex-error for builder and io errors

* Use flex-error for rpc errors

* Use flex-error for protobuf errors

* Use flex-error for abci

* Fix test_bisection_no_witness_left

* Fix build errors in all-features

* Fix failing tests

* Fix more failures

* Fix tungstenite error under wasm target

* Fix incoming_fixtures test

* Fix conflict

* Update flex-error to v0.4.0

* set std feature in flex-error instead of individual crates

* Add flex-error patch to tools/Cargo.toml

* Use published version of flex-error v0.4.1

* Enable flex-error/eyre_tracer feature by default

* Add .changelog entry (#940)

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* flex-error: resolve conflicts with `master` (#945)

* Implement full-duplex secret connection (#938)

* Implement thread-safe cloning of a secret connection

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Expand documentation for SecretConnection on threading considerations

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Extract peer construction into its own method

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add test for cloned SecretConnection

This adds a `TcpStream`-based test for parallelizing operations on a
`SecretConnection`. I used `TcpStream` instead of the buffered reader in
the other tests because it wasn't feasible to implement the `TryClone`
trait for that buffered pipe implementation.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add more messages to test

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Expand comment for clarity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add .changelog entry

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Restore half-duplex operations

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Extract encrypt/decrypt fns as independent methods

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove unnecessary trait bounds

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Extract send/receive state

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Extract read/write functionality as standalone methods

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add logic to facilitate splitting SecretConnection into its sending and receiving halves

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Restore split SecretConnection test using new semantics

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update changelog entry

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update docs for `SecretConnection`

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Condense error reporting

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Extract TryClone trait into its own crate

As per the discussion at
#938 (comment),
this extracts the `TryClone` trait into a new crate called
`tendermint-std-ext` in the `std-ext` directory.

This new crate is intended to contain any code that we need that extends
the Rust standard library.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Reorder imports

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Assert validation regardless of debug build

This introduces the internal encryption assertions at runtime regardless
of build type. This may introduce a small performance hit, but it's
probably worth it to ensure correctness.

Effectively this is keeping an eye on the code in the
`encrypt_and_write` fn to ensure its correctness.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove remote_pubkey optionality from sender/receiver halves

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update SecretConnection docs with comment content

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix doc link to TryClone trait

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix doc link to TryClone trait

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add docs on SecretConnection failures and connection integrity

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Synchronize sending/receiving failures to comply with crypto algorithm constraints

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Rename try_split method to split for SecretConnection

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove redundant field name prefixes

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix broken link in docs

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix recent clippy errors on `master` (#941)

* Fix needless borrows in codebase

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Ignore needless collect warning (we do actually seem to need it)

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove trailing semicolon in macro to fix docs compiling

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove unnecessary macros

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Correct error messages

Signed-off-by: Thane Thomson <connect@thanethomson.com>

Co-authored-by: Thane Thomson <thane@informal.systems>
Co-authored-by: Thane Thomson <connect@thanethomson.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants