Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add experimental yamux support (networking) #7437

Merged
merged 23 commits into from
Sep 20, 2023
Merged

Add experimental yamux support (networking) #7437

merged 23 commits into from
Sep 20, 2023

Conversation

StefanBratanov
Copy link
Contributor

@StefanBratanov StefanBratanov commented Aug 21, 2023 •
edited
Loading

PR Description

  • Upgrade jvm-libp2p to 1.0.1 with improved yamux support in comparison to 1.0.0
  • Increase maxBufferedConnection writes to 150 MiB as a band-aid fix untill more optimized yamux backpressure is implemented in jvm-libp2p
  • Introduce --Xp2p-yamux-enabled to enable/disable yamux. Default will be false until tested extensively and newer version of libp2p is released with improved backpressure.
  • Rename MplexFirewall to MuxFirewall and made it compatible with yamux frames.
  • Add jitpack.io repo. It is required for transitive dependency in libp2p (multibase-java )
  • Explicitly use netty-codec-http and netty-handler because they are runtime dependencies in libp2p (not exposed by the library)

Fixed Issue(s)

N/A

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

  • I thought about adding a changelog entry, and added one if I deemed necessary.

@StefanBratanov StefanBratanov changed the title Update jvm-libp2p to 1.0.0 Add yamux support (libp2p) Aug 21, 2023
@StefanBratanov StefanBratanov changed the title Add yamux support (libp2p) Add yamux support (networking) Aug 22, 2023
@StefanBratanov StefanBratanov marked this pull request as ready for review August 22, 2023 21:36
@StefanBratanov StefanBratanov changed the title Add yamux support (networking) Add experimental yamux support (networking) Aug 23, 2023
Nashatyrev
Nashatyrev approved these changes Aug 23, 2023
View reviewed changes
Copy link
Contributor

@Nashatyrev Nashatyrev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

gradle/versions.gradle Outdated
@@ -58,10 +58,8 @@ dependencyManagement {
}

dependencySet(group: 'io.netty', version: '4.1.96.Final') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Libp2p has a bit older version 4.1.90 but it seems ok. May upgrade Libp2p on the next release

@jxs jxs mentioned this pull request Aug 23, 2023
Open
@StefanBratanov StefanBratanov marked this pull request as draft September 6, 2023 16:57
@StefanBratanov StefanBratanov force-pushed the update_jvm_libp2p branch 2 times, most recently from 45e531a to 0527fcd Compare September 14, 2023 10:27
@StefanBratanov StefanBratanov marked this pull request as ready for review September 16, 2023 14:02
rolfyone
rolfyone reviewed Sep 20, 2023
View reviewed changes
networking/p2p/src/main/java/tech/pegasys/teku/networking/p2p/libp2p/LibP2PNetworkBuilder.java Outdated

// yamux MUST take precedence during negotiation
if (config.isYamuxEnabled()) {
// TODO: setting maxBufferedConnectionWrites to 150 MiB to handle overflowing the write
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

outstanding todo? is there a ticket?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@StefanBratanov It looks like the default config is already 150MiB? https://github.com/libp2p/jvm-libp2p/pull/325/files

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, added one #7532 will reference it in the comment

Copy link
Contributor Author

@StefanBratanov StefanBratanov Sep 20, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@StefanBratanov It looks like the default config is already 150MiB? https://github.com/libp2p/jvm-libp2p/pull/325/files

We closed that PR, agreed it's better to override it in Teku instead for the time being, since 150 MiB was excessive as a default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah my bad, didn't notice it was closed instead of merged ;)

rolfyone
rolfyone approved these changes Sep 20, 2023
View reviewed changes
Copy link
Contributor

@rolfyone rolfyone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM just a todo question

@StefanBratanov StefanBratanov enabled auto-merge (squash) September 20, 2023 04:33
@StefanBratanov StefanBratanov merged commit bb082cc into Consensys:master Sep 20, 2023
11 of 14 checks passed
@StefanBratanov StefanBratanov deleted the update_jvm_libp2p branch September 20, 2023 05:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lucassaldanha lucassaldanha lucassaldanha left review comments

@rolfyone rolfyone rolfyone approved these changes

@Nashatyrev Nashatyrev Nashatyrev approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@StefanBratanov @lucassaldanha @rolfyone @Nashatyrev