Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade libp2p to v0.47.0 #3491

Conversation

divagant-martian
Copy link
Collaborator

@divagant-martian divagant-martian commented Aug 22, 2022

Issue Addressed

Upgrades libp2p to v0.47.0. Addresses the deprecation of some traits and parts of the derive macro that force handling behaviour events above the swarm level. See libp2p swarm release notes

Contains #3495 and #3497 so better to give those a pass to make reading this easier.

This upgrade also requires an explicit/manual installation of protoc. See 0.47.0 release notes

Proposed Changes

  • last version of libp2p requires an explicit/manual installation of the protobuf compiler (protoc) so this PR:
    • Modifies the installation instructions.
    • Modifies the build from source instructions.
    • Updates Dockerfile to reflect this.
    • Updates CI using an action. I've seen Rate limits here so we might want to change this.
  • Create a minimal Behaviour struct that derives NetworkBehaviour. This generates the event BehaviourEvent.
  • The above generated BehaviourEvent requires debug, so it's now derived for the sub-behaviour events that did not implement it.
  • Renames the old Behaviour to Network.
  • Moves the swarm inside the (new) Network.
  • Since handling behaviour events now is done by polling the swarm, all the NetworkBehaviourEventProcess implementation are moved to functions in the inject_<sub-behaviour>_event fashion.
  • Merges the Libp2pEvent into BehaviourEvent (simply by moving the two extra events NewListenAddr and ZeroListeners). This is renamed to NetworkEvent
  • Merges polling the swarm with polling the old behaviour.
  • After all this changes the old service.rs file had only utility functions for building the swarm, so this file is moved and renamed to service/utils.rs. Noting this change here explicitly because git does not recognize the file as moved but as deleted.
  • Updates tests accordingly.
  • The libp2p upgrade carries a Prometheus client upgrade which solves RUSTSEC-2022-0040

Additional Info

na

@divagant-martian divagant-martian added the work-in-progress PR is a work-in-progress label Aug 22, 2022
@divagant-martian divagant-martian changed the base branch from stable to unstable August 22, 2022 16:45
@divagant-martian divagant-martian force-pushed the networking-version-upgrades branch 2 times, most recently from 775c866 to 03e016a Compare August 23, 2022 20:00
@michaelsproul michaelsproul changed the title upgrage libp2p upgrade libp2p Aug 24, 2022
@michaelsproul
Copy link
Member

I couldn't help updating the title - we can change it back to upg-rage if that was an accurate description of writing this PR 😁

@divagant-martian
Copy link
Collaborator Author

lmao what an accurate title description the previous one was @michaelsproul. Thanks for fixing it

@divagant-martian divagant-martian changed the base branch from unstable to libp2p-v0.47.0-upgrade August 26, 2022 20:06
@divagant-martian divagant-martian marked this pull request as ready for review August 26, 2022 20:41
@divagant-martian divagant-martian changed the title upgrade libp2p Upgrade libp2p to v0.47.0 Aug 26, 2022
@divagant-martian divagant-martian added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Aug 26, 2022
Copy link
Member

@AgeManning AgeManning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I like this. The majority of code changes seem to be just re-organising structure.

It all looks fine to me.

Ok((network, network_globals))
}

// TODO: docs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might as well, while we are here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in 1e5694f

pub fn poll_network(&mut self, cx: &mut Context) -> Poll<NetworkEvent<AppReqId, TSpec>> {
while let Poll::Ready(Some(swarm_event)) = self.swarm.poll_next_unpin(cx) {
let maybe_event = match swarm_event {
SwarmEvent::Behaviour(behaviour_event) => match behaviour_event {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new libp2p now makes us do this manually? Thats kinda annoying

But your soln here does look cleaner.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this is essentially the core of the required change

Copy link
Member

@AgeManning AgeManning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, lets dooo it

@AgeManning AgeManning merged commit d1a9635 into sigp:libp2p-v0.47.0-upgrade Sep 7, 2022
bors bot pushed a commit that referenced this pull request Sep 12, 2022
## Issue Addressed

Upgrades libp2p to v.0.47.0. This is the compilation of
- [x] #3495 
- [x] #3497 
- [x] #3491 
- [x] #3546 
- [x] #3553 

Co-authored-by: Age Manning <Age@AgeManning.com>
bors bot pushed a commit that referenced this pull request Sep 29, 2022
## Issue Addressed

Upgrades libp2p to v.0.47.0. This is the compilation of
- [x] #3495 
- [x] #3497 
- [x] #3491 
- [x] #3546 
- [x] #3553 

Co-authored-by: Age Manning <Age@AgeManning.com>
bors bot pushed a commit that referenced this pull request Oct 24, 2022
## Issue Addressed

I missed this from #3491. peers were being banned at the behaviour level only. The identify errors are explained by this as well

## Proposed Changes

Add banning and unbanning 

## Additional Info

Befor,e having tests that catch this was hard because the swarm was outside the behaviour. We could now have tests that prevent something like this in the future
macladson pushed a commit to macladson/lighthouse that referenced this pull request Jan 5, 2023
## Issue Addressed

I missed this from sigp#3491. peers were being banned at the behaviour level only. The identify errors are explained by this as well

## Proposed Changes

Add banning and unbanning 

## Additional Info

Befor,e having tests that catch this was hard because the swarm was outside the behaviour. We could now have tests that prevent something like this in the future
Woodpile37 pushed a commit to Woodpile37/lighthouse that referenced this pull request Jan 6, 2024
## Issue Addressed

Upgrades libp2p to v.0.47.0. This is the compilation of
- [x] sigp#3495 
- [x] sigp#3497 
- [x] sigp#3491 
- [x] sigp#3546 
- [x] sigp#3553 

Co-authored-by: Age Manning <Age@AgeManning.com>
Woodpile37 pushed a commit to Woodpile37/lighthouse that referenced this pull request Jan 6, 2024
## Issue Addressed

I missed this from sigp#3491. peers were being banned at the behaviour level only. The identify errors are explained by this as well

## Proposed Changes

Add banning and unbanning 

## Additional Info

Befor,e having tests that catch this was hard because the swarm was outside the behaviour. We could now have tests that prevent something like this in the future
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Networking ready-for-review The code is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants