create standard bencher config we can use for performance comparison/regression testing

[porcuquine]

Description

We are landing many optimizations of different size and scope. We need a way of analyzing each individually for performance (memory, cpu, wall clock) — as well as for potentially comparing curated combinations. At the very least, we need a standard way of checking what effect an incremental change believed to be an improvement has.

Fortunately, we have the basic mechanism for this in place already:

• modify bench.config.toml to add a more realistic zigzag.
  Ideally, we would use:
  • m: 5
  • expansion: 8
  • layers: 10
  • taper: 1.0 / 3.0
  • taper_layers: 7
  • sloth: 0
  • partitions: 1 (for now)
  • challenges: might be as low as 1 or 2, but shoot for higher. Note 'total_challenges' computed in output.
  • size: ideally, 1 GiB to start, but as small as 256 MiB if necessary.

To test this, create a PR on https://github.com/filecoin-project/rust-proofs-infra. You will need to reference your branch in that PR (see instruction in README for that repo).

You may need to contact infra to get access to the results.

Acceptance criteria

It is possible to get a standard zigzag benchmark, with wall clock, total cpu, and max memory consumption for any branch, using the rust-proofs-infra mechanism but without doing anything special to the branch itself.

Ideally, this PR would also involve harvesting results and putting them in the PR description in what can become a standard format. This should include enough information for those with access to reference the raw data themselves. (I believe SHA1 is in the meta-data. If not, should be added on rust-proofs-infra. There should be a short section in our README (or probably CONTRIBUTING) explaining the procedure for benchmarking and reporting in PRs.

Risks + pitfalls

Actually testing this will be a little slow. The bench machine has, I believe, 32GiB so it might take a little fiddling to find a configuration that is mostly realistic but succeeds on the machine.

Where to begin

bench.config.toml and rust-proofs-infra should get you going.

[schomatis]

Depends on https://github.com/filecoin-project/rust-proofs-infra/issues/22.

[schomatis]

Also depends on https://github.com/filecoin-project/rust-proofs-infra/issues/17.

[schomatis]

See #577 for a very rough script that attempts to go in this direction.