Load test bitswap retrieval with Piece Directory

[dirkmc]

Depends on

• Add tracing to piece directory implementations #889
• Integrate Piece Directory into booster-bitswap #893

---

Create a tool (or script) that load tests bitswap retrieval by targetting a specific provider with a predefined set of cids to fetch, with configurable concurrency factor. We want to start building intuition on what the limits of retrieval are for Boost at the moment.

Given that currently our devnets support only small sector sizes (8MiB), this tool would initially have to target a mainnet provider, such as sofiaminer / https://filfox.info/en/address/f0127896, until we add 32GiB sector support with fake proofs to the devnet.

During and after the load test we should be able to extract insight from the metrics/tracing systems on performance of various internal components

• total number. rate and latency of requests to Boost -- on API layer
• total number, rate and latency of requests to Boost -- on Piece directory
• total number, rate and latency of requests to Boost -- piece reader handlers
• total number, rate and latency of requests to Lotus -- piece reader handlers

---

This task is somewhat open-ended and attempts to tie together the metrics/tracing effort with the devnet effort and create a framework to evaluate Boost retrievals consistently.

---

Related:

• Add support for 32GiB sectors to devnet #902

[jacobheun]

I am currently testing this against the working local index branch in docker and leveraging #1143 to add some latency in front of bitswap. The latter part is likely not necessary for this but better mimics a real work environment. Preliminary results show there might be a leak, I'll post back here with pprofs and metrics graphs after a few more runs.

[jacobheun]

After prolonged load the in use memory allocation is actually stable, the increase I was seeing before was just increased memory allocation to the process due to being under load. We do have reports of increased memory for users testing bitswap in the current RC, so I am going to repeat my tests against the current RC that does not contain the local index directory (piece directory) code.