Problem
The current benchmark table is a flat list with one row per fixture/tool/method/command combination. With ferrflow alone that's already 48 rows (4 fixtures × 3 methods × 4 commands). Adding changesets, semantic-release, and release-please will push this to 192+ rows — unreadable in a PR comment or release note.
Current format:
| Fixture |
Tool |
Method |
Command |
Median |
Memory |
| mono-large |
ferrflow |
binary |
check |
189.8ms |
21.5 MB |
| mono-large |
ferrflow |
binary |
release-dry-run |
187.5ms |
21.4 MB |
| mono-large |
ferrflow |
binary |
tag |
1781.8ms |
15.1 MB |
| mono-large |
ferrflow |
binary |
version |
12.3ms |
10.9 MB |
| ... |
... |
... |
... |
... |
... |
Proposal
Group by method (binary / npm / docker), then use a pivot layout where commands become columns:
Binary
| Fixture |
Tool |
check |
release |
tag |
version |
Peak RSS |
| single |
ferrflow |
14.7ms |
13.7ms |
13.0ms |
10.8ms |
12.4 MB |
| mono-small |
ferrflow |
15.1ms |
15.0ms |
15.6ms |
11.2ms |
12.6 MB |
| mono-medium |
ferrflow |
23.2ms |
23.6ms |
70.5ms |
11.6ms |
13.0 MB |
| mono-large |
ferrflow |
189.8ms |
187.5ms |
1781.8ms |
12.3ms |
21.5 MB |
This reduces 48 rows → 12 rows for one tool, and 192 rows → 48 for four tools. Each table stays compact enough for PR comments.
When multiple tools are present, they sort together per fixture for direct comparison:
| Fixture |
Tool |
check |
release |
tag |
version |
Peak RSS |
| mono-large |
ferrflow |
189.8ms |
187.5ms |
1781.8ms |
12.3ms |
21.5 MB |
| mono-large |
semantic-release |
... |
... |
... |
... |
... |
| mono-large |
changesets |
... |
... |
... |
... |
... |
Scope
scripts/run.sh — main table generation
scripts/format-release.sh — release notes summary
- Possibly
scripts/compare.sh if regression output uses the same format
Problem
The current benchmark table is a flat list with one row per fixture/tool/method/command combination. With ferrflow alone that's already 48 rows (4 fixtures × 3 methods × 4 commands). Adding changesets, semantic-release, and release-please will push this to 192+ rows — unreadable in a PR comment or release note.
Current format:
Proposal
Group by method (binary / npm / docker), then use a pivot layout where commands become columns:
Binary
This reduces 48 rows → 12 rows for one tool, and 192 rows → 48 for four tools. Each table stays compact enough for PR comments.
When multiple tools are present, they sort together per fixture for direct comparison:
Scope
scripts/run.sh— main table generationscripts/format-release.sh— release notes summaryscripts/compare.shif regression output uses the same format