New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestions for improving consensus rate in Tendermint core #1426

Closed
ragjunk opened this Issue Apr 5, 2018 · 7 comments

Comments

Projects
None yet
4 participants
@ragjunk

ragjunk commented Apr 5, 2018

This is a suggestion for improvement

Here's a proposal, courtesy of @StorecoinTeam, to minimize the peer-to-peer communication overhead in the Tendermint core. It leverages Tendermint's ABCI protocol to implement a "shared mempool" where the nodes collaboratively "order" incoming transactions. During the consensus round, the proposer dequeues the transactions from the shared mempool. The proposal is available here - https://github.com/StorecoinProject/Basil

@cloudinertia

This comment has been minimized.

Show comment
Hide comment
@cloudinertia

cloudinertia Apr 6, 2018

In order to maintain "shared mempool", there still needed tendermint-like consensus for verify it.

But such malicious behavior is easy to detect because the malicious peer would produce a longer sequence in a given period of time (such as between consensus rounds) compared to the honest peers. (from https://github.com/StorecoinProject/Basil)

I think it's too weak statement. As a common user, I want more noble reasoning such as "shared mempool is about N times faster than original one" or "from logical argument we can trust shared mempool at certain condition".

cloudinertia commented Apr 6, 2018

In order to maintain "shared mempool", there still needed tendermint-like consensus for verify it.

But such malicious behavior is easy to detect because the malicious peer would produce a longer sequence in a given period of time (such as between consensus rounds) compared to the honest peers. (from https://github.com/StorecoinProject/Basil)

I think it's too weak statement. As a common user, I want more noble reasoning such as "shared mempool is about N times faster than original one" or "from logical argument we can trust shared mempool at certain condition".

@ragjunk

This comment has been minimized.

Show comment
Hide comment
@ragjunk

ragjunk Apr 6, 2018

In order to maintain "shared mempool", there still needed tendermint-like consensus for verify it.

Please see the approach described in the proposal, which uses a sequence of sha256 hashes to "order" transactions. Each validator node runs this sequence independent of others. As each node receives transactions, it "embeds" it in the sequence and writes it to the shared mempool. Each entry in the shared mempool is also signed by respective validators, so it can be associated with who created that entry. The proposer can simply walk the sequence to retrieve the ordered transactions from multiple validators. This requires no additional consensus.

I think it's too weak statement.

Assume each of the N validator nodes receives one transaction. In the original TM core, each node broadcasts the transactions to all other nodes, resulting in (N^2 - N) messages. For example, if N = 5, there will be 20 messages. In this approach, each validator nodes writes to shared mempool, so there will be only N messages. Of course, there is a "local computation" required to create the sequence, which is not there in the original TM core. But unreliable communication is generally what drags the performance down.

While not mentioned in the proposal, this behavior can be completely optional. If the specified ABCI interfaces are implemented, TM core uses it; otherwise it falls back to existing behavior.

ragjunk commented Apr 6, 2018

In order to maintain "shared mempool", there still needed tendermint-like consensus for verify it.

Please see the approach described in the proposal, which uses a sequence of sha256 hashes to "order" transactions. Each validator node runs this sequence independent of others. As each node receives transactions, it "embeds" it in the sequence and writes it to the shared mempool. Each entry in the shared mempool is also signed by respective validators, so it can be associated with who created that entry. The proposer can simply walk the sequence to retrieve the ordered transactions from multiple validators. This requires no additional consensus.

I think it's too weak statement.

Assume each of the N validator nodes receives one transaction. In the original TM core, each node broadcasts the transactions to all other nodes, resulting in (N^2 - N) messages. For example, if N = 5, there will be 20 messages. In this approach, each validator nodes writes to shared mempool, so there will be only N messages. Of course, there is a "local computation" required to create the sequence, which is not there in the original TM core. But unreliable communication is generally what drags the performance down.

While not mentioned in the proposal, this behavior can be completely optional. If the specified ABCI interfaces are implemented, TM core uses it; otherwise it falls back to existing behavior.

@ebuchman

This comment has been minimized.

Show comment
Hide comment
@ebuchman

ebuchman Apr 6, 2018

Contributor

I don't understand why it's called "Shared Mempool".

Currently, everyone broadcasts all mempool txs, so with enough time, everyone will see the same txs, though its possible they'll see them in different orders.

In this proposal, it looks like the nodes don't gossip the mempool transactions at all. Why would the resulting mempool be called "shared" if the transactions aren't being shared?

Contributor

ebuchman commented Apr 6, 2018

I don't understand why it's called "Shared Mempool".

Currently, everyone broadcasts all mempool txs, so with enough time, everyone will see the same txs, though its possible they'll see them in different orders.

In this proposal, it looks like the nodes don't gossip the mempool transactions at all. Why would the resulting mempool be called "shared" if the transactions aren't being shared?

@ragjunk

This comment has been minimized.

Show comment
Hide comment
@ragjunk

ragjunk Apr 7, 2018

Currently, everyone broadcasts all mempool txs, so with enough time, everyone will see the same txs

The proposal was to avoid the broadcasts, which are expensive, especially when you have a large number of validator nodes spread across the globe.

In this proposal, it looks like the nodes don't gossip the mempool transactions at all

Right. Instead of pooling transactions into their "local" mempool, they write to the "shared" mempool, while still being able to preserve the transaction order. This proposal avoids gossip to update all peers on incoming transactions.

ragjunk commented Apr 7, 2018

Currently, everyone broadcasts all mempool txs, so with enough time, everyone will see the same txs

The proposal was to avoid the broadcasts, which are expensive, especially when you have a large number of validator nodes spread across the globe.

In this proposal, it looks like the nodes don't gossip the mempool transactions at all

Right. Instead of pooling transactions into their "local" mempool, they write to the "shared" mempool, while still being able to preserve the transaction order. This proposal avoids gossip to update all peers on incoming transactions.

@ebuchman

This comment has been minimized.

Show comment
Hide comment
@ebuchman

ebuchman Apr 7, 2018

Contributor

Why is it called a shared mempool if nothing is being shared?

Contributor

ebuchman commented Apr 7, 2018

Why is it called a shared mempool if nothing is being shared?

@ragjunk

This comment has been minimized.

Show comment
Hide comment
@ragjunk

ragjunk Apr 8, 2018

Why is it called a shared mempool if nothing is being shared?

Transactions received by all validators are shared via this mechanism and hence the name. In other words, instead of broadcasting/gossiping the transactions among all the other peers, the validators simply order the transactions in the shared mempool, thus minimizing the total number of messages required to update all the validators. Each receiving node writes the transaction to the shared mempool, so if each node receives say, m transactions, there will be only m*N messages.

See the proposal for details.

ragjunk commented Apr 8, 2018

Why is it called a shared mempool if nothing is being shared?

Transactions received by all validators are shared via this mechanism and hence the name. In other words, instead of broadcasting/gossiping the transactions among all the other peers, the validators simply order the transactions in the shared mempool, thus minimizing the total number of messages required to update all the validators. Each receiving node writes the transaction to the shared mempool, so if each node receives say, m transactions, there will be only m*N messages.

See the proposal for details.

@ebuchman

This comment has been minimized.

Show comment
Hide comment
@ebuchman

ebuchman May 14, 2018

Contributor

Thanks for this - in an effort to clean up our issues, I'm going to close this for now.

Later this year we will revisit options for optimizing the consensus.

Thanks!

Contributor

ebuchman commented May 14, 2018

Thanks for this - in an effort to clean up our issues, I'm going to close this for now.

Later this year we will revisit options for optimizing the consensus.

Thanks!

@ebuchman ebuchman closed this May 14, 2018

@xla xla removed the post-launch label Jul 18, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment