adding bench tests for WNLA

distributed-lab · May 21, 2024 · 10b5f57 · 10b5f57
1 parent 6500490
commit 10b5f57
Showing 1 changed file with 20 additions and 10 deletions.
diff --git a/macbook-m3-pro-36GB-bench-result.txt b/macbook-m3-pro-36GB-bench-result.txt
@@ -6,17 +6,27 @@ test tests::wnla_works ... ignored
 
 test result: ok. 0 passed; 0 failed; 3 ignored; 0 measured; 0 filtered out; finished in 0.00s
 
-prove                   time:   [14.576 ms 14.653 ms 14.736 ms]
-                        change: [-0.4837% +0.2403% +0.9140%] (p = 0.50 > 0.05)
+prove                   time:   [14.438 ms 14.443 ms 14.449 ms]
+                        change: [-0.0859% -0.0360% +0.0145%] (p = 0.18 > 0.05)
                         No change in performance detected.
-Found 18 outliers among 100 measurements (18.00%)
-  1 (1.00%) high mild
-  17 (17.00%) high severe
+Found 9 outliers among 100 measurements (9.00%)
+  7 (7.00%) high mild
+  2 (2.00%) high severe
 
-verify                  time:   [3.9424 ms 3.9434 ms 3.9445 ms]
-                        change: [-0.6983% -0.6274% -0.5620%] (p = 0.00 < 0.05)
-                        Change within noise threshold.
-Found 5 outliers among 100 measurements (5.00%)
+verify                  time:   [3.7385 ms 3.7405 ms 3.7426 ms]
+                        change: [-1.6307% -1.4039% -1.2033%] (p = 0.00 < 0.05)
+                        Performance has improved.
+Found 3 outliers among 100 measurements (3.00%)
+  3 (3.00%) high mild
+
+prove WNLA              time:   [661.30 µs 661.50 µs 661.72 µs]
+Found 10 outliers among 100 measurements (10.00%)
+  1 (1.00%) low mild
+  5 (5.00%) high mild
+  4 (4.00%) high severe
+
+verify WNLA             time:   [349.06 µs 349.93 µs 350.98 µs]
+Found 6 outliers among 100 measurements (6.00%)
   4 (4.00%) high mild
-  1 (1.00%) high severe
+  2 (2.00%) high severe