Bash script to help benchmark the compiler

[xunilrj]

Description

This PR introduces a script that compiles a sway project across multiple commits, and generates an HTML report to help analysis of compilation time and binary size.

The report is 100% autonomous, only depending on CDN scripts, and can be served as it is by any HTTP server. There are two main features worthy of being noted: a pivot table, and the data table.

The pivot table is configured by default to show the average compilation time across multiple commits, like the example below.

Below it, one finds the data table used, with links to Github issues and commits.

The script help can be printed by invoking it without any arguments

> test/bench.sh

 -n|--qty          how many commits will be used
 -b|--branch       which branch will be used. Defaults to "master"
 -p|--path         path to sway project to use
 -c|--clean        what to clean. Options:
     "results" - will clean all csv, txt files
     "exe"     - will clean all compiled executables
     "all"     - all the above
 -o|--open         open the final report using the default browser

A good first invocation example is:

> test/bench.sh -n 2 -p test/src/e2e_vm_tests/test_programs/should_pass/language/main_args/main_args_empty/ -o
master d93b "fix: fund default signer account in ci (#6055)" [compiling] [benchmark] [bin size] ok
master 5389 "Add examples on how to import storage types to the book (#6051)" [compiling] [benchmark] [bin size] ok
Warning: Cache at "/home/xunilrj/.cache/sway-bench" is using 108M of your disk space
Opening in existing browser session.

The script tries to be gentle on printing to the terminal, so details are redirected to $HOME/.cache/sway-bench/log.txt, which can be tail -f if needed.

So the only feedback is that for each commit, it prints:

• which branch is being used;
• its hash;
• its message;
• and which step is running: compiling forc, running the benchmark and calculating the binary size.

The script runs incrementally. So it never redo a step if its results exist in the cache. A second consecutive run should not do anything. This is particularly important when compiling all the multiple versions of forc, which can take quite a long time!

The option -o will open the browser pointing to the report after everything is finished.

An important detail is disk usage. At the end of the script, it reminds you how much space is being used. Each forc version is consuming more than 50MB. One can easily clean this with --clean. It asks permission before deleting anything, but it is always important to double-check if it is not doing something crazy.

> test/bench.sh --clean all

If you have a specific branch you want to test, you can try

> test/bench.sh -n 1 -p test/src/e2e_vm_tests/test_programs/should_pass/language/main_args/main_args_empty/ --branch xunilrj/configurables-faster-access

Suggestions and improvements are welcomed.

Checklist

• I have linked to any relevant issues.
• I have commented my code, particularly in hard-to-understand areas.
• I have updated the documentation where relevant (API docs, the reference, and the Sway book).
  • If my change requires substantial documentation changes, I have requested support from the DevRel team
• I have added tests that prove my fix is effective or that my feature works.
• I have added (or requested a maintainer to add) the necessary Breaking* or New Feature labels where relevant.
• I have done my best to ensure that my PR adheres to the Fuel Labs Code Review Standards.
• I have requested a review from the relevant team or maintainers.