Create test to show libp2p's performance compared to http

[BigLep]

Done Criteria

We have benchmarks that shows the performance of libp2p using various transports/muxers in real world conditions compared to something standard in the web2 world like HTTP.

Why Important

Questions come in on, "why use libp2p and protocols build on top instead of something like http? Isn't http faster?" Ideally we should have data showing where libp2p transports/muxers stand.

User/Customer

Developers determining whether libp2p will be placing performance overhead, which can affect their decision to adopt libp2p as a whole.

Notes

1. Initial focus is on go-libp2p but ideally other libp2p implementations can be added.
2. https://github.com/filecoin-project/data-transfer-benchmark is likely a good place to start from.
3. Minimum set of results is:

• http
• http over libp2p using TCP/Yamux
• http over libp2p using QUIC

4. When tests are run locally, we need to make sure an appropriate MTU is set: limit packet size to typical MTUs used on the public internet testground/testground#1385
5. Ideally though tests can be run against two distinct nodes sending traffic over the internet, which is the real world conditions we want to compare against. feat: Add Nix Builder + Add remote runner example plan testground/testground#1425 or EPIC: Testground remote:docker and remote:exec runners testground/testground#1392 is needed for this.

[BigLep]

@marten-seemann : FYI that I created this issue. Please adjust if anything is off.

(For context this came out of a private FIL Slack conversation: https://filecoinproject.slack.com/archives/C03HQ6C0TFT/p1657643516791279 . This conversation didn't need to be private, and any more correspondence will be public and can then be linked to here.)

(Public thread in #libp2p-implementers: https://filecoinproject.slack.com/archives/C03K82MU486/p1658794025325099 )

[marten-seemann]

This mostly looks good.

2. https://github.com/filecoin-project/data-transfer-benchmark is likely a good place to start from.

Yes, but we'll probably want to end up with something that doesn't pull in dependencies from outside of libp2p. Most importantly, our test will not exercise graphsync.

Minimum set of results is:

• http
• http over libp2p using TCP/Yamux
• http over libp2p using Quic

We'll use TCP instead of HTTP, for multiple reasons

1. It's not clear which HTTP we're comparing to: HTTP1.1, HTTP/2, H3?
2. The HTTP layer on top of libp2p is very simple. It doesn't add any overhead.

We should also benchmark mplex, given that it's still widely used. No expectation of good performance here, but it would be nice to generate the numbers nevertheless.

4. When tests are run locally, we need to make sure an appropriate MTU is set: limit packet size to typical MTUs used on the public internet testground/testground#1385

This is a worthwhile change, but there's probably little value in running transport performance tests locally to begin with.

[BigLep]

We don't have a canoncial spot for this work stream yet, but dumping in some more recent notes:

What is performance

As reminded by Marten, whenever we talk about performance, we need to be clear and educating about it:

1. Handshake performance, i.e. “how many RTTs do I have to wait until I have a fully functioning libp2p connection”. For the vast majority of use cases, this is way more interesting than throughput. This is also a nice way to show why we like QUIC so much.
2. Throughput, i.e. “how many GB / s can I transfer over the connection”. This is very tricky to measure in a meaningful way.
3. CPU / memory overhead. Probably mostly a concern for TCP-based transports, as for QUIC, a libp2p stream is just a QUIC stream and your QUIC stack does all the heavy lifting for you (this might not apply to the quinn-wrapper in rust-libp2p).

What this should look like

Marco wrote some thoughts in a wip branch in testplans: https://github.com/libp2p/test-plans/blob/marco/perf/perf-dashboard/README.md

Before we start doing coding, I want to make sure we sketch the end results. Similar how the resulting interop test matrix (example) gave the interoperability project focus, I want to make sure we have the end state sketched out for the performance benchmarks.