chore(bench-compilation): improve output, bench check etc.

[dpc]

No description provided.

[bradleystachurski]

Might be unique to my setup, but just bench-compilation fails

Full check debug: Error! Results might be invalid.
error: Recipe `bench-compilation` failed on line 123 with exit code 127

[dpc]

Might be unique to my setup, but just bench-compilation fails

Can you bash -x scripts/bench-compilation.sh? What's happening there? @bradleystachurski

[Kodylow]

got this error when testing locally:

  Downloaded 16 crates (6.8 MB) in 2.93s (largest was `linux-raw-sys` at 1.5 MB)
kody@Kodys-MacBook-Pro-2.local 
------------------------------ 
OS: macOS 14.2 23C64 arm64 
Host: Mac14,10 
Kernel: 23.2.0 
Uptime: 24 days, 8 hours, 19 mins 
Packages: 231 (brew), 94 (nix-user) 
Shell: zsh 5.9 
Resolution: 3456x2234 
DE: Aqua 
WM: Quartz Compositor 
WM Theme: Blue (Dark) 
Terminal: vscode 
CPU: Apple M2 Pro 
GPU: Apple M2 Pro 
Memory: 9856MiB / 16384MiB 

Date: 2024-03-07
Commit: 39d5eeeed4
                        total   user    sys
Full check debug: Error! Results might be invalid.
error: Recipe `bench-compilation` failed on line 123 with exit code 1

then tried with bash per previous comment and got this:

(nix:nix-shell-env) Kodys-MacBook-Pro-2:fedimint kody$ bash -x scripts/bench-compilation.sh
+ set -euo pipefail
+ trap on_error ERR
+ trap on_exit EXIT
++ git rev-parse --show-toplevel
+ root=/Users/kody/Documents/github/fedi_stuff/fedimint
+ export CARGO_BUILD_TARGET_DIR=/Users/kody/Documents/github/fedi_stuff/fedimint/target-comp-bench
+ CARGO_BUILD_TARGET_DIR=/Users/kody/Documents/github/fedi_stuff/fedimint/target-comp-bench
+ unset RUSTC_WRAPPER
+ cargo fetch
+ nix run nixpkgs#neofetch -- --stdout
kody@Kodys-MacBook-Pro-2.local 
------------------------------ 
OS: macOS 14.2 23C64 arm64 
Host: Mac14,10 
Kernel: 23.2.0 
Uptime: 24 days, 8 hours, 20 mins 
Packages: 231 (brew), 94 (nix-user) 
Shell: zsh 5.9 
Resolution: 3456x2234 
DE: Aqua 
WM: Quartz Compositor 
WM Theme: Blue (Dark) 
Terminal: vscode 
CPU: Apple M2 Pro 
GPU: Apple M2 Pro 
Memory: 9909MiB / 16384MiB 

++ date +%Y-%m-%d
+ echo 'Date: 2024-03-07'
Date: 2024-03-07
++ git rev-parse --short HEAD
+ echo 'Commit: 39d5eeeed4'
Commit: 39d5eeeed4
+ time_pipe_path=/Users/kody/Documents/github/fedi_stuff/fedimint/target-comp-bench/time.out
+ time_fmt='\t%e\t%U\t%S'
+ echo -e '                   \ttotal\tuser\tsys'
                        total   user    sys
+ for profile in dev release
+ for command in check build
+ rm -Rf /Users/kody/Documents/github/fedi_stuff/fedimint/target-comp-bench
+ mkdir -p /Users/kody/Documents/github/fedi_stuff/fedimint/target-comp-bench
+ profile_human=dev
+ '[' dev = dev ']'
+ profile_human=debug
+ echo -n 'Full check debug: '
Full check debug: + command time '--format=\t%e\t%U\t%S' -o /Users/kody/Documents/github/fedi_stuff/fedimint/target-comp-bench/time.out -- cargo check --profile dev -q
++ on_error
++ echo 'Error! Results might be invalid.'
Error! Results might be invalid.
+ on_exit
+ rm -Rf /Users/kody/Documents/github/fedi_stuff/fedimint/target-comp-bench
(nix:nix-shell-env) Kodys-MacBook-Pro-2:fedimint kody$ 

[dpc]

@Kodylow Is your local source in a clean state that compiles? You can remove 1>/dev/null 2>/dev/null from the script to see if cargo throws any errors.

[Kodylow]

just cleaned, rebuilt, and checked those all passed still throws the same error

[dpc]

@Kodylow @bradleystachurski I pushed a change that should print out any errors it might find. Please retry. bash -x scripts/bench-compilation.sh

[bradleystachurski]

Command error:
++ cat /home/nix/fedimint/target-nix/cmd.out
scripts/bench-compilation.sh: line 53: time: command not found

command is causing an issue finding time.

nix@lenovo-m75q-fedimint-2:~/fedimint$ time echo ''


real    0m0.000s
user    0m0.000s
sys     0m0.000s
nix@lenovo-m75q-fedimint-2:~/fedimint$ command time echo ''
bash: time: command not found

[dpc]

Huh. I thought time is part of coreutils etc...

[dpc]

@bradleystachurski Here, have some time.

[bradleystachurski]

Adding time to the dev shell fixed on my end 👍

Benchmarks look good locally, but I think we should remove the ci timeout commit from this PR.

dce295b (#4500)

[dpc]

Sorry, decided to fix that padding after all. It just didn't spark joy.

Date: 2024-03-08
Commit: 9388403f40
                       total    user     sys
Full  check   debug:   35.22  466.85   40.46
Incr  check   debug:    2.87    3.62    2.22
Full  build   debug:   63.77 1282.28   89.12
Incr  build   debug:    9.44   17.53    8.63
Full  check release:   31.63  258.01   35.32
Incr  check release:    9.52   14.69    2.27
Full  build release:   86.41 1831.95   81.06
Incr  build release:   48.47 1031.01   28.06

[bradleystachurski]

[nit] Do we need gawk? Works fine locally without

[dpc]

Merge conflict. Of course.