Updated benchmark suite

[b-fg]

• Cost in tables
• Plots
• compare.jl can now be run with command line arguments which admit global expression
• Improved juliup functionality in benchmark.sh
• Fixed measure bug

[b-fg]

This is ready to be tested! @weymouth mind giving it a try on Windows?

[weymouth]

$ juliaup st
 Default  Channel      Version                        Update                    
-------------------------------------------------------------------------------------------------------
          1.8.0-beta3  1.8.0-beta3+0.x64.w64.mingw32                            
          beta         1.10.0+0.x64.w64.mingw32       Update to 1.11.0-rc1+0.x64.w64.mingw32 available
       *  release      1.10.0+0.x64.w64.mingw32       Update to 1.10.4+0.x64.w64.mingw32 available

admin@DESKTOP-4014140 MINGW64 ~/Documents/GitHub/WaterLily/benchmark (fast-measure)
$ sh benchmark.sh
The latest version of Julia in the `release` channel is 1.10.4+0.x64.w64.mingw32. You currently have `1.10.0+0.x64.w64.mingw32` installed. Run:

  juliaup update

in your terminal shell to install Julia 1.10.4+0.x64.w64.mingw32 and update the `release` channel to that version.
--------------------------------------
Running benchmark tests for:
 - Julia:        1.10.0
 - Backends:     Array CuArray
 - CPU threads:  1 4
 - Cases:        tgv jelly
 - Size:         6,7 5,6
 - Sim. steps:   100 100
 - Data type:    Float32 Float32
--------------------------------------

Julia 1.10.0 benchmarks
Updating environment to Julia 1.10.0
`1.10.0` is not installed. Please run `juliaup add 1.10.0` to install channel or version.
Running: julia +1.10.0 --project=/c/Users/admin/Documents/GitHub/WaterLily/benchmark --startup-file=no -t 1 /c/Users/admin/Documents/GitHub/WaterLily/benchmark/benchmark.jl --backend=Array --cases=["tgv","jelly"] --log2p=[(6,7),(5,6)] --max_steps=[100,100] --ftype=[Float32,Float32]
`1.10.0` is not installed. Please run `juliaup add 1.10.0` to install channel or version.
Running: julia +1.10.0 --project=/c/Users/admin/Documents/GitHub/WaterLily/benchmark --startup-file=no -t 4 /c/Users/admin/Documents/GitHub/WaterLily/benchmark/benchmark.jl --backend=Array --cases=["tgv","jelly"] --log2p=[(6,7),(5,6)] --max_steps=[100,100] --ftype=[Float32,Float32]
`1.10.0` is not installed. Please run `juliaup add 1.10.0` to install channel or version.
Running: julia +1.10.0 --project=/c/Users/admin/Documents/GitHub/WaterLily/benchmark --startup-file=no /c/Users/admin/Documents/GitHub/WaterLily/benchmark/benchmark.jl --backend=CuArray --cases=["tgv","jelly"] --log2p=[(6,7),(5,6)] --max_steps=[100,100] --ftype=[Float32,Float32]
`1.10.0` is not installed. Please run `juliaup add 1.10.0` to install channel or version.
All done!

[weymouth]

It doesn't see the 1.10.0+... as 1.10.0?

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

see 10 files with indirect coverage changes

[weymouth]

Is there another test I can try with this? Maybe using a windows GPU tomorrow?

[b-fg]

Yeah, that'd be great. Tomorrow I will also to some final tests and it should be ready to merge.

[b-fg]

I added support to test different WaterLily versions with e.g. -w "a271054a e22ad41". Also did some final benchmarks for Julia 1.10.4 vs 1.11.0, and there is a consistent speedup on GPU (10%) and a large speedup for serial execution (>70%)! cc for ref: #153

Benchmarks

Benchmark environment: tgv sim_step! (max_steps=100)
▶ log2p = 6
┌─────────┬───────────┬────────────┬───────────┬─────────────┬────────┬──────────┬──────────────────┬──────────┐
│ Backend │ WaterLily │   Julia    │ Precision │ Allocations │ GC [%] │ Time [s] │ Cost [ns/DOF/dt] │ Speed-up │
├─────────┼───────────┼────────────┼───────────┼─────────────┼────────┼──────────┼──────────────────┼──────────┤
│  CPUx01 │   a271054 │     1.10.4 │   Float32 │       78733 │   0.00 │     9.10 │           346.99 │     1.00 │
│  CPUx01 │   a271054 │ 1.11.0-rc1 │   Float32 │       78734 │   0.00 │     5.22 │           199.10 │     1.74 │
│  CPUx04 │   a271054 │     1.10.4 │   Float32 │     2274514 │   0.00 │     3.18 │           121.27 │     2.86 │
│  CPUx04 │   a271054 │ 1.11.0-rc1 │   Float32 │     2274515 │   0.70 │     3.19 │           121.78 │     2.85 │
│  CPUx08 │   a271054 │     1.10.4 │   Float32 │     3555070 │   0.00 │     3.33 │           127.01 │     2.73 │
│  CPUx08 │   a271054 │ 1.11.0-rc1 │   Float32 │     3555071 │   1.05 │     4.06 │           154.71 │     2.24 │
│    CUDA │   a271054 │     1.10.4 │   Float32 │     2671159 │   0.00 │     0.64 │            24.34 │    14.26 │
│    CUDA │   a271054 │ 1.11.0-rc1 │   Float32 │     2466144 │   0.00 │     0.58 │            22.26 │    15.59 │
└─────────┴───────────┴────────────┴───────────┴─────────────┴────────┴──────────┴──────────────────┴──────────┘
▶ log2p = 7
┌─────────┬───────────┬────────────┬───────────┬─────────────┬────────┬──────────┬──────────────────┬──────────┐
│ Backend │ WaterLily │   Julia    │ Precision │ Allocations │ GC [%] │ Time [s] │ Cost [ns/DOF/dt] │ Speed-up │
├─────────┼───────────┼────────────┼───────────┼─────────────┼────────┼──────────┼──────────────────┼──────────┤
│  CPUx01 │   a271054 │     1.10.4 │   Float32 │       70606 │   0.00 │    73.16 │           348.87 │     1.00 │
│  CPUx01 │   a271054 │ 1.11.0-rc1 │   Float32 │       70607 │   0.00 │    39.12 │           186.56 │     1.87 │
│  CPUx04 │   a271054 │     1.10.4 │   Float32 │     2021882 │   0.00 │    18.60 │            88.68 │     3.93 │
│  CPUx04 │   a271054 │ 1.11.0-rc1 │   Float32 │     2021883 │   0.00 │    19.06 │            90.90 │     3.84 │
│  CPUx08 │   a271054 │     1.10.4 │   Float32 │     3159482 │   0.00 │    19.29 │            91.99 │     3.79 │
│  CPUx08 │   a271054 │ 1.11.0-rc1 │   Float32 │     3159483 │   0.17 │    22.95 │           109.42 │     3.19 │
│    CUDA │   a271054 │     1.10.4 │   Float32 │     2347006 │   0.00 │     3.11 │            14.84 │    23.51 │
│    CUDA │   a271054 │ 1.11.0-rc1 │   Float32 │     2158507 │   0.00 │     2.97 │            14.17 │    24.63 │
└─────────┴───────────┴────────────┴───────────┴─────────────┴────────┴──────────┴──────────────────┴──────────┘
Benchmark environment: jelly sim_step! (max_steps=100)
▶ log2p = 5
┌─────────┬───────────┬────────────┬───────────┬─────────────┬────────┬──────────┬──────────────────┬──────────┐
│ Backend │ WaterLily │   Julia    │ Precision │ Allocations │ GC [%] │ Time [s] │ Cost [ns/DOF/dt] │ Speed-up │
├─────────┼───────────┼────────────┼───────────┼─────────────┼────────┼──────────┼──────────────────┼──────────┤
│  CPUx01 │   a271054 │     1.10.4 │   Float32 │      196731 │   0.00 │     7.72 │           588.72 │     1.00 │
│  CPUx01 │   a271054 │ 1.11.0-rc1 │   Float32 │      196732 │   0.00 │     4.27 │           325.85 │     1.81 │
│  CPUx04 │   a271054 │     1.10.4 │   Float32 │     5549128 │   0.00 │     4.53 │           345.92 │     1.70 │
│  CPUx04 │   a271054 │ 1.11.0-rc1 │   Float32 │     5549129 │   0.77 │     4.19 │           319.45 │     1.84 │
│  CPUx08 │   a271054 │     1.10.4 │   Float32 │     8644684 │   3.09 │     5.10 │           388.96 │     1.51 │
│  CPUx08 │   a271054 │ 1.11.0-rc1 │   Float32 │     8644685 │   1.24 │     5.33 │           406.75 │     1.45 │
│    CUDA │   a271054 │     1.10.4 │   Float32 │     6542147 │   0.00 │     1.49 │           113.79 │     5.17 │
│    CUDA │   a271054 │ 1.11.0-rc1 │   Float32 │     6083377 │   0.73 │     1.34 │           102.16 │     5.76 │
└─────────┴───────────┴────────────┴───────────┴─────────────┴────────┴──────────┴──────────────────┴──────────┘
▶ log2p = 6
┌─────────┬───────────┬────────────┬───────────┬─────────────┬────────┬──────────┬──────────────────┬──────────┐
│ Backend │ WaterLily │   Julia    │ Precision │ Allocations │ GC [%] │ Time [s] │ Cost [ns/DOF/dt] │ Speed-up │
├─────────┼───────────┼────────────┼───────────┼─────────────┼────────┼──────────┼──────────────────┼──────────┤
│  CPUx01 │   a271054 │     1.10.4 │   Float32 │      230016 │   0.00 │    58.29 │           555.88 │     1.00 │
│  CPUx01 │   a271054 │ 1.11.0-rc1 │   Float32 │      230087 │   0.00 │    30.01 │           286.16 │     1.94 │
│  CPUx04 │   a271054 │     1.10.4 │   Float32 │     6552597 │   0.00 │    20.40 │           194.58 │     2.86 │
│  CPUx04 │   a271054 │ 1.11.0-rc1 │   Float32 │     6552668 │   0.19 │    21.56 │           205.65 │     2.70 │
│  CPUx08 │   a271054 │     1.10.4 │   Float32 │    10222545 │   0.67 │    21.47 │           204.78 │     2.71 │
│  CPUx08 │   a271054 │ 1.11.0-rc1 │   Float32 │    10222616 │   0.32 │    24.17 │           230.53 │     2.41 │
│    CUDA │   a271054 │     1.10.4 │   Float32 │     7787222 │   0.00 │     4.64 │            44.24 │    12.56 │
│    CUDA │   a271054 │ 1.11.0-rc1 │   Float32 │     7259394 │   0.44 │     4.16 │            39.69 │    14.00 │
└─────────┴───────────┴────────────┴───────────┴─────────────┴────────┴──────────┴──────────────────┴──────────┘

This PR is now ready to merge.