Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New gossip_net backend using libp2p #3988

Merged
merged 220 commits into from
Feb 10, 2020
Merged

New gossip_net backend using libp2p #3988

merged 220 commits into from
Feb 10, 2020

Conversation

emberian
Copy link
Contributor

@emberian emberian commented Nov 25, 2019

This is the revival of the tmp/cmr/net2 branch rebased onto develop.

Some important user-facing changes:

  • No separate discovery/communication/etc ports. One port for all public daemon communications.
  • Automatic port forwarding with UPnP. If your local network supports UPnP, there should be no configuration required.
  • Local peer discovery. If your local network supports mDNS broadcast, coda daemons will automatically discover each other. This includes several daemons on the same machine- no more building peer lists!
  • New libp2p keypairs. These are managed the same as our key pairs with secret_file. Without configuration, key pairs are ephemeral and will disappear when the daemon restarts. (TODO: should we instead persist the keypair? does it matter for non-infrastructure?)

Some important internal changes:

  • All daemon-daemon connections are now authenticated and confidential.
  • Connections are no longer transient and per-request. Individual requests get multiplexed as their own stream over the one connection between the peers. This is analogous to HTTP/2. Outgoing connections will appear to originate from the libp2p listening port, vs some transient port.

Outstanding details:

Other changes:

  • Rips out the last vestiges of old membership, which aren't in use.
  • The connection info in envelopes is much more accurate now. We shouldn't start trusting it just yet due to some future vagaries around relaying.
  • bump nixpkgs version

Future improvements:

  • IPv6. There's a hardcoded IPv4 assumption in the helper around IP filtering.
  • Investigate libp2p autorelay. This should help nodes in restrictive networks achieve better connectivity, but has a host of problems.
  • Intelligent request routing. I believe we can use the "provider" feature to, at the very least, only send eg sync/bootstrap requests to nodes who believe themselves to be in sync. There are other options.

@emberian emberian changed the title New gossip_net backend New gossip_net backend using libp2p Nov 25, 2019
 Conflicts:
	.circleci/config.yml
	.circleci/config.yml.jinja
	.gitattributes
	CODEOWNERS
	frontend/website/docs/developers/directory-structure.md
	frontend/website/docs/glossary.md
	src/app/cli/src/tests/coda_archive_node_test.ml
	src/app/cli/src/tests/coda_batch_payment_test.ml
	src/app/cli/src/tests/coda_bootstrap_test.ml
	src/app/cli/src/tests/coda_change_snark_worker_test.ml
	src/app/cli/src/tests/coda_delegation_test.ml
	src/app/cli/src/tests/coda_peers_test.ml
	src/app/cli/src/tests/coda_process.ml
	src/app/cli/src/tests/coda_processes.ml
	src/app/cli/src/tests/coda_receipt_chain_test.ml
	src/app/cli/src/tests/coda_restart_node_test.ml
	src/app/cli/src/tests/coda_restarts_and_txns_holy_grail.ml
	src/app/cli/src/tests/coda_shared_prefix_test.ml
	src/app/cli/src/tests/coda_transitive_peers_test.ml
	src/app/cli/src/tests/coda_txns_and_restart_non_producers.ml
	src/app/cli/src/tests/coda_worker.ml
	src/app/cli/src/tests/coda_worker_testnet.ml
	src/lib/coda_lib/coda_lib.ml
	src/lib/coda_lib/dune
	src/lib/consensus/proof_of_stake.ml
	src/lib/daemon_rpcs/types.ml
	src/lib/transition_frontier_controller/transition_frontier_controller.ml
@emberian emberian changed the base branch from release/0.0.12-beta to develop February 6, 2020 00:31
@bkase bkase merged commit 16b8125 into develop Feb 10, 2020
@bkase bkase deleted the feature/net2 branch February 10, 2020 22:36
@ghost-not-in-the-shell ghost-not-in-the-shell restored the feature/net2 branch July 16, 2020 01:27
@ghost-not-in-the-shell ghost-not-in-the-shell deleted the feature/net2 branch July 16, 2020 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants