Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

X-chain SDK gossip #2490

Merged
merged 116 commits into from Dec 22, 2023
Merged

X-chain SDK gossip #2490

merged 116 commits into from Dec 22, 2023

Conversation

joshua-kim
Copy link
Contributor

@joshua-kim joshua-kim commented Dec 15, 2023

Why this should be merged

This PR introduces p2p SDK based tx gossip to the X-chain. This includes pull and push gossip of transactions (under a new message format) while maintaining compatibility with the prior push gossip mechanism.

How this works

  • Makes *txs.Tx implement the Gossipable interface
  • Creates a gossip mempool that wraps the prior mempool with a bloom filter and (optional) verification checks.
  • Separates the AVM config into it's own file to better handle the new networking based configs
  • Modifies the existing network struct to pass all AppRequest, AppResponse, and AppRequestFailed messages to the p2p SDK network implementation.
  • Modifies the existing network struct to pass AppGossip messages that can't be parsed with the prior format to the p2p SDK network implementation.
  • Deprecates the prior push gossip mechanism.

How this was tested

  • CI
  • Tested on fuji
  • Tested on a local network while explicitly dropping the legacy message format
  • Tested on a local network while explicitly dropping the new message format

joshua-kim and others added 24 commits December 15, 2023 11:27
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Co-authored-by: Stephen Buttolph <stephen@avalabs.org>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Co-authored-by: Stephen Buttolph <stephen@avalabs.org>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Co-authored-by: Stephen Buttolph <stephen@avalabs.org>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Stephen Buttolph <stephen@avalabs.org>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Co-authored-by: Stephen Buttolph <stephen@avalabs.org>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
@joshua-kim joshua-kim force-pushed the xchain-gossip branch 2 times, most recently from d974b7e to c58063d Compare December 15, 2023 16:44
Signed-off-by: Joshua Kim <20001595+joshua-kim@users.noreply.github.com>
@StephenButtolph StephenButtolph marked this pull request as ready for review December 22, 2023 14:49
Base automatically changed from tx-verifier to dev December 22, 2023 15:28
}

func (g *gossipMempool) AddVerified(tx *txs.Tx) error {
if err := g.Mempool.Add(tx); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check if tx is already in the mempool?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AddVerified is expected to error if the tx is already in the mempool (which is what happens here)

StephenButtolph and others added 3 commits December 22, 2023 15:28
Co-authored-by: Dhruba Basu <7675102+dhrubabasu@users.noreply.github.com>
Signed-off-by: Stephen Buttolph <stephen@avalabs.org>
@StephenButtolph StephenButtolph added this pull request to the merge queue Dec 22, 2023
Merged via the queue into dev with commit 33f7411 Dec 22, 2023
17 checks passed
@StephenButtolph StephenButtolph deleted the xchain-gossip branch December 22, 2023 21:34
@joshua-kim joshua-kim mentioned this pull request Dec 22, 2023
txID := tx.ID()
n.txPushGossiper.Add(tx)
if err := n.txPushGossiper.Gossip(ctx); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we try to better batch these gossip invocations to reduce the number of messages we push (no more than one every x ms)? Or is that handled in the SDK?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT we would batch if we didn't invoke manually:

// Every calls [Gossip] every [frequency] amount of time.
func Every(ctx context.Context, log logging.Logger, gossiper Gossiper, frequency time.Duration) {
ticker := time.NewTicker(frequency)
defer ticker.Stop()
for {
select {
case <-ticker.C:
if err := gossiper.Gossip(ctx); err != nil {
log.Warn("failed to gossip", zap.Error(err))
}
case <-ctx.Done():
log.Debug("shutting down gossip")
return
}
}
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah - this is kind of an independent suggestion to this PR (as this is the same behavior as before)... But we could play around with batching

if err := n.issueTx(tx); err != nil {
// IssueTx attempts to add a tx to the mempool, after verifying it. If the tx is
// added to the mempool, it will attempt to push gossip the tx to random peers
// in the network using both the legacy and p2p SDK.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to just key off of support based on the handshake?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we already get version in the Connected message, so it may not be that bad but could be wrong.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use AppGossip for this, so we don't have the ability to select who exactly we are sending the messages to for push gossip. We could filter the pull gossip to only send it to newer nodes if we cared to optimize this during the time between this release and the network upgrade.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Durango durango fork
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

4 participants