Skip to content

Conversation

@kmiller68
Copy link
Contributor

Right now the CLI summary reports its metrics as "Total". I assumed at first glace this meant sum of all the categories. Since that's what the "Total-time" metric is reporting. If the label was "Overall", I think this would be more obvious. The last "Total Time" is now "Overall Wall-Time" to be more consistent with the line items' equivalent report.

I also reordered the line item "Total-time" to be before "Wall-Time" because I find that to be a slightly more intuitive order.

Before this change:

Running 8bitbench-wasm:
8bitbench-wasm First-Score                      23.66 pts
8bitbench-wasm Worst-Score                      31.79 pts
8bitbench-wasm Average-Score                    32.19 pts
8bitbench-wasm Score                            28.93 pts
8bitbench-wasm First-Time                      211.36 ms
8bitbench-wasm Worst-Time                      157.26 ms
8bitbench-wasm Average-Time                    155.35 ms
8bitbench-wasm Wall-Time                      2390.92 ms
8bitbench-wasm Total-Time                      523.97 ms

Total:
Total First-Score                              164.07 pts
Total Worst-Score                              306.75 pts
Total Average-Score                            427.19 pts
Total Stdlib-Score                              13.10 pts
Total MainRun-Score                              2.89 pts
Total First-Time                                31.58 ms
Total Worst-Time                                16.97 ms
Total Average-Time                              12.16 ms
Total Stdlib-Time                              381.82 ms
Total MainRun-Time                            1732.12 ms

Total Score                                    254.28 pts
Total Time                                    93008.80 ms

After:

Running 8bitbench-wasm:
8bitbench-wasm First-Score                      23.14 pts
8bitbench-wasm Worst-Score                      31.58 pts
8bitbench-wasm Average-Score                    32.05 pts
8bitbench-wasm Score                            28.61 pts
8bitbench-wasm First-Time                      216.06 ms
8bitbench-wasm Worst-Time                      158.35 ms
8bitbench-wasm Average-Time                    155.98 ms
8bitbench-wasm Total-Time                      530.39 ms
8bitbench-wasm Wall-Time                      2404.08 ms

Overall:
Overall First-Score                            171.40 pts
Overall Worst-Score                            330.65 pts
Overall Average-Score                          444.21 pts
Overall Stdlib-Score                            16.04 pts
Overall MainRun-Score                            2.80 pts
Overall First-Time                              30.24 ms
Overall Worst-Time                              15.70 ms
Overall Average-Time                            11.68 ms
Overall Stdlib-Time                            311.64 ms
Overall MainRun-Time                          1785.38 ms

Overall Score                                  268.03 pts
Overall Wall-Time                             87146.46 ms

Right now the CLI summary reports its metrics as "Total". I assumed at
first glace this meant sum of all the categories. Since that's what
the "Total-time" metric is reporting. If the label was "Overall", I
think this would be more obvious. The last "Total Time" is now
"Overall Wall-Time" to be more consistent with the line items'
equivalent report.

I also reordered the line item "Total-time" to be before "Wall-Time"
because I find that to be a slightly more intuitive order.

Before this change:
```
Running 8bitbench-wasm:
8bitbench-wasm First-Score                      23.66 pts
8bitbench-wasm Worst-Score                      31.79 pts
8bitbench-wasm Average-Score                    32.19 pts
8bitbench-wasm Score                            28.93 pts
8bitbench-wasm First-Time                      211.36 ms
8bitbench-wasm Worst-Time                      157.26 ms
8bitbench-wasm Average-Time                    155.35 ms
8bitbench-wasm Wall-Time                      2390.92 ms
8bitbench-wasm Total-Time                      523.97 ms

Total:
Total First-Score                              164.07 pts
Total Worst-Score                              306.75 pts
Total Average-Score                            427.19 pts
Total Stdlib-Score                              13.10 pts
Total MainRun-Score                              2.89 pts
Total First-Time                                31.58 ms
Total Worst-Time                                16.97 ms
Total Average-Time                              12.16 ms
Total Stdlib-Time                              381.82 ms
Total MainRun-Time                            1732.12 ms

Total Score                                    254.28 pts
Total Time                                    93008.80 ms
```

After:
```
Running 8bitbench-wasm:
8bitbench-wasm First-Score                      23.14 pts
8bitbench-wasm Worst-Score                      31.58 pts
8bitbench-wasm Average-Score                    32.05 pts
8bitbench-wasm Score                            28.61 pts
8bitbench-wasm First-Time                      216.06 ms
8bitbench-wasm Worst-Time                      158.35 ms
8bitbench-wasm Average-Time                    155.98 ms
8bitbench-wasm Total-Time                      530.39 ms
8bitbench-wasm Wall-Time                      2404.08 ms

Overall:
Overall First-Score                            171.40 pts
Overall Worst-Score                            330.65 pts
Overall Average-Score                          444.21 pts
Overall Stdlib-Score                            16.04 pts
Overall MainRun-Score                            2.80 pts
Overall First-Time                              30.24 ms
Overall Worst-Time                              15.70 ms
Overall Average-Time                            11.68 ms
Overall Stdlib-Time                            311.64 ms
Overall MainRun-Time                          1785.38 ms

Overall Score                                  268.03 pts
Overall Wall-Time                             87146.46 ms
```
@netlify
Copy link

netlify bot commented Oct 22, 2025

Deploy Preview for webkit-jetstream-preview ready!

Name Link
🔨 Latest commit f4bcde4
🔍 Latest deploy log https://app.netlify.com/projects/webkit-jetstream-preview/deploys/68f8e639ee6619000883b758
😎 Deploy Preview https://deploy-preview-216--webkit-jetstream-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@kmiller68
Copy link
Contributor Author

I didn't change the console.log or benchmark-total-time-score for the measureTotalTimeAsSubtest because I wasn't sure if anyone was relying on that. Happy to include those if no one is.

@camillobruni
Copy link
Contributor

This will require some gymnastics on our internal CQ to keep the labels stable – but I like the change. Total was always confusing.

@kmiller68
Copy link
Contributor Author

Ah, is there a reason you parse the stdout rather than the simple JSON output?

@kmiller68 kmiller68 merged commit 3ec29fe into WebKit:main Oct 23, 2025
10 checks passed
@camillobruni
Copy link
Contributor

our infra settled on manual stdout parsing since that was originally more convenient with all the different runners 🤷 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants