Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Node discovery via DNS #7129

Merged
merged 42 commits into from
Jun 5, 2024

Conversation

usmansaleem
Copy link
Member

@usmansaleem usmansaleem commented May 22, 2024

PR description

Fold Tuweni dns discovery module into Besu p2p discovery module. The Tuweni dns-discovery was Kotlin code using kotlin co-routines. It has been translated (and adapted) in Java. Utilized vertx worker verticle to launch dns daemon in a separate thread outside event loop. Rest of the method protocol is exactly same at the moment.

The screenshot shows mainnet peer connection for four nodes which were configured with SNAP and CHECKPOINT.

Screenshot 2024-06-06 at 10 08 47 AM

Fixed Issue(s)

Thanks for sending a pull request! Have you done the following?

  • Checked out our contribution guidelines?
  • Considered documentation and added the doc-change-required label to this PR if updates are required.
  • Considered the changelog and included an update if required.
  • For database changes (e.g. KeyValueSegmentIdentifier) considered compatibility and performed forwards and backwards compatibility tests

Locally, you can run these tests to catch failures early:

  • unit tests: ./gradlew build
  • acceptance tests: ./gradlew acceptanceTest
  • integration tests: ./gradlew integrationTest
  • reference tests: ./gradlew ethereum:referenceTests:referenceTests

 -- Use besu-dns-discovery which is a fork of Tuweni dns discovery

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
CHANGELOG.md Outdated Show resolved Hide resolved
macfarla and others added 4 commits May 22, 2024 16:46
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: Usman Saleem <usman@usmans.info>
…nsensys_dns_discovery

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
@shemnon
Copy link
Contributor

shemnon commented May 22, 2024

I'm curious as to why this was ported to a separate Consensys lib and not folded directly into Besu. Are there other projects that use this library? Is it because it's written on Kotlin? It's only 5 production files and 2 test files if wiring in a kotlin compiler step isn't desired (surely less effort than maintaining an ongoing deployment). I't also makes it more difficult to alter the classes if features such as streaming the nodes instead of querying them all in one go is desired.

@usmansaleem
Copy link
Member Author

@shemnon Hi Danno, the initial thought was just to port it as it is to make further changes and include it as a library, however, you've raised valid points, I concur that folding these classes (the dns-discovery) into besu should not be too difficult. I am not aware of any other projects using the Tuweni dns-discovery module.

@macfarla any further thoughts?

@usmansaleem usmansaleem marked this pull request as draft May 22, 2024 23:19
@macfarla
Copy link
Contributor

@shemnon Hi Danno, the initial thought was just to port it as it is to make further changes and include it as a library, however, you've raised valid points, I concur that folding these classes (the dns-discovery) into besu should not be too difficult. I am not aware of any other projects using the Tuweni dns-discovery module.

@macfarla any further thoughts?

yeah fair point @shemnon if it's easier long term I'm not opposed to folding it into Besu. Initial thinking was we could still revert to tuweni libs again if they got updated but that also may not eventuate.

@shemnon
Copy link
Contributor

shemnon commented May 23, 2024

As a bit more context, I'm looking at folding the tuweni-bytes libraries back into besu (undoing #215 in effect). My main reason is a tighter optmiization loop when speeding up the EVM as well as being able to purpose-build some of the bytes implementation for speed. Coordinating these changes across repos creates significant friction.

 -- migrated from Tuweni

Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
@usmansaleem usmansaleem changed the title feat: Use besu-dns-discovery feat: enr DNS discovery without Tuweni May 26, 2024
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
@usmansaleem usmansaleem marked this pull request as ready for review May 28, 2024 00:17
Signed-off-by: Usman Saleem <usman@usmans.info>
@usmansaleem usmansaleem requested a review from macfarla May 28, 2024 01:31
…rticle

Signed-off-by: Usman Saleem <usman@usmans.info>
@usmansaleem usmansaleem changed the title feat: enr DNS discovery without Tuweni feat: Node discovery via DNS May 28, 2024
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

// Adapted from https://github.com/tmio/tuweni and licensed under Apache 2.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this sufficient attribution as far as the apache license is concerned?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is. Although we may have to update our NOTICE file with Tuweni's NOTICE file. Our NOTICE file is empty though. Also, we didn't do a verbatim copy of the code, we adapted and converted it into Java language from Kotlin language.

https://opensource.stackexchange.com/questions/8717/how-to-attribute-code-from-a-project-under-apache-v2-in-a-new-project

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we add all contributors info to the notice going forward?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if we are "reproducing" or "including" code from an Apache license project that contains a NOTICE file, it ought to be included in our NOTICE file as well. I am not sure whether we had to include NOTICE attributions from Vertx and other libraries that are included in binary form. @non-fungible-nelson FYI for your input as well.

ethereum/p2p/src/test/resources/log4j2-test.xml Outdated Show resolved Hide resolved
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
Copy link
Contributor

@shemnon shemnon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

…very

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
Signed-off-by: Usman Saleem <usman@usmans.info>
NOTICE Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

…very

Signed-off-by: Usman Saleem <usman@usmans.info>
…very

Signed-off-by: Usman Saleem <usman@usmans.info>
@usmansaleem usmansaleem merged commit e4902af into hyperledger:main Jun 5, 2024
38 checks passed
@usmansaleem usmansaleem deleted the consensys_dns_discovery branch June 5, 2024 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants