Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new mempool benchmarks for a complex pool #17292

Merged
merged 1 commit into from Nov 1, 2019

Conversation

@JeremyRubin
Copy link
Contributor

JeremyRubin commented Oct 28, 2019

This PR is related to #17268.

It adds a mempool stress test which makes a really big complicated tx graph, and then, similar to mempool_eviction test, trims the size.

The test setup is to make 100 original transactions with Rand(10)+2 outputs each.

Then, 800 times:

we create a new transaction with Rand(10) + 1 parents that are randomly sampled from all existing transactions (with unspent outputs). From each such parent, we then select Rand(remaining outputs) +1 50% of the time, or 1 outputs 50% of the time.

Then, we trim the size to 3/4. Then we trim it to just a single transaction.

This creates, hopefully, a big bundle of transactions with lots of complex structure, that should really put a strain on the mempool graph algorithms.

This ends up testing both the descendant and ancestor tracking.

I don't love that the test is "unstable". That is, in order to compare this test to another, you really can't modify any of the internal state because it will have a different order of invocations of the deterministic randomness. However, it certainly suffices for comparing branches.

@JeremyRubin

This comment has been minimized.

Copy link
Contributor Author

JeremyRubin commented Oct 29, 2019

For those curious, this is a graph of the transactions that are being tested here. The red ones are the "parentless" ones that we create (along with the fake parent I put in to make the graph render nicely.

This graph doesn't show the weights for how many outputs are being consumed by a child from a parent, an inbound edge simply indicates any amount of consumption.

Txn Graph

@MarcoFalke

This comment has been minimized.

Copy link
Member

MarcoFalke commented Oct 29, 2019

Concept ACK! I was waiting for this

@instagibbs

This comment has been minimized.

Copy link
Member

instagibbs commented Oct 29, 2019

concept ACK! moar data

@JeremyRubin

This comment has been minimized.

Copy link
Contributor Author

JeremyRubin commented Oct 29, 2019

Moar data:

I ran #17268 against master across a number of paramters for the number of transactions.

image

Transactions Master Epoch MemPool Speedup
800 0.26 0.122783 2.117556991
2000 1.81 0.7989 2.265615221
5000 11.92 5.14931 2.314873255
8000 32.73 13.21 2.477668433
10000 54.1336 22.5658 2.398922263
12500 87.64 39.2 2.235714286
17500 182.268 86.34 2.11104934
20000 248.16 118.3 2.097717667

What's interesting about the testing setup is if the only modification you make is the number of transactions, it is "monotonic" that is, G(n) is a subgraph of G(n+1) for all n. Adjusting other parameters will not have the same sort of effect, unfortunately, as the construction is not stable.

@jamesob

This comment has been minimized.

Copy link
Member

jamesob commented Oct 29, 2019

Concept ACK

@JeremyRubin

This comment has been minimized.

Copy link
Contributor Author

JeremyRubin commented Oct 29, 2019

Not as useful to look at, but this is the structure with 10,000 nodes and 800. If anyone wants the svg (fully zoomable), can share.

This is made using the twopi shell like layout. The other layout engines choke with this many nodes.

smaller_hairball

bighairball

MarcoFalke added a commit that referenced this pull request Nov 1, 2019
b0c774b Add new mempool benchmarks for a complex pool (Jeremy Rubin)

Pull request description:

  This PR is related to #17268.

  It adds a mempool stress test which makes a really big complicated tx graph, and then, similar to mempool_eviction test, trims the size.

  The test setup is to make 100 original transactions with Rand(10)+2 outputs each.

  Then, 800 times:

  we create a new transaction with Rand(10) + 1 parents that are randomly sampled from all existing transactions (with unspent outputs). From each such parent, we then select Rand(remaining outputs) +1 50% of the time, or 1 outputs 50% of the time.

  Then, we trim the size to 3/4. Then we trim it to just a single transaction.

  This creates, hopefully, a big bundle of transactions with lots of complex structure, that should really put a strain on the mempool graph algorithms.

  This ends up testing both the descendant and ancestor tracking.

  I don't love that the test is "unstable". That is, in order to compare this test to another, you really can't modify any of the internal state because it will have a different order of invocations of the deterministic randomness. However, it certainly suffices for comparing branches.

Top commit has no ACKs.

Tree-SHA512: cabe96b849b9885878e20eec558915e921d49e6ed1e4b011b22ca191b4c99aa28930a8b963784c9adf78cc8b034a655513f7a0da865e280a1214ae15ebb1d574
@MarcoFalke MarcoFalke merged commit b0c774b into bitcoin:master Nov 1, 2019
2 checks passed
2 checks passed
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
sidhujag added a commit to syscoin/syscoin that referenced this pull request Nov 2, 2019
b0c774b Add new mempool benchmarks for a complex pool (Jeremy Rubin)

Pull request description:

  This PR is related to bitcoin#17268.

  It adds a mempool stress test which makes a really big complicated tx graph, and then, similar to mempool_eviction test, trims the size.

  The test setup is to make 100 original transactions with Rand(10)+2 outputs each.

  Then, 800 times:

  we create a new transaction with Rand(10) + 1 parents that are randomly sampled from all existing transactions (with unspent outputs). From each such parent, we then select Rand(remaining outputs) +1 50% of the time, or 1 outputs 50% of the time.

  Then, we trim the size to 3/4. Then we trim it to just a single transaction.

  This creates, hopefully, a big bundle of transactions with lots of complex structure, that should really put a strain on the mempool graph algorithms.

  This ends up testing both the descendant and ancestor tracking.

  I don't love that the test is "unstable". That is, in order to compare this test to another, you really can't modify any of the internal state because it will have a different order of invocations of the deterministic randomness. However, it certainly suffices for comparing branches.

Top commit has no ACKs.

Tree-SHA512: cabe96b849b9885878e20eec558915e921d49e6ed1e4b011b22ca191b4c99aa28930a8b963784c9adf78cc8b034a655513f7a0da865e280a1214ae15ebb1d574
@JeremyRubin JeremyRubin added this to Testing in Mempool improvements Jan 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Mempool improvements
General Testing
5 participants
You can’t perform that action at this time.