Skip to content

Restructure benchmark results table to reduce row count #76

@BryanFRD

Description

@BryanFRD

Problem

The current benchmark table is a flat list with one row per fixture/tool/method/command combination. With ferrflow alone that's already 48 rows (4 fixtures × 3 methods × 4 commands). Adding changesets, semantic-release, and release-please will push this to 192+ rows — unreadable in a PR comment or release note.

Current format:

Fixture Tool Method Command Median Memory
mono-large ferrflow binary check 189.8ms 21.5 MB
mono-large ferrflow binary release-dry-run 187.5ms 21.4 MB
mono-large ferrflow binary tag 1781.8ms 15.1 MB
mono-large ferrflow binary version 12.3ms 10.9 MB
... ... ... ... ... ...

Proposal

Group by method (binary / npm / docker), then use a pivot layout where commands become columns:

Binary

Fixture Tool check release tag version Peak RSS
single ferrflow 14.7ms 13.7ms 13.0ms 10.8ms 12.4 MB
mono-small ferrflow 15.1ms 15.0ms 15.6ms 11.2ms 12.6 MB
mono-medium ferrflow 23.2ms 23.6ms 70.5ms 11.6ms 13.0 MB
mono-large ferrflow 189.8ms 187.5ms 1781.8ms 12.3ms 21.5 MB

This reduces 48 rows → 12 rows for one tool, and 192 rows → 48 for four tools. Each table stays compact enough for PR comments.

When multiple tools are present, they sort together per fixture for direct comparison:

Fixture Tool check release tag version Peak RSS
mono-large ferrflow 189.8ms 187.5ms 1781.8ms 12.3ms 21.5 MB
mono-large semantic-release ... ... ... ... ...
mono-large changesets ... ... ... ... ...

Scope

  • scripts/run.sh — main table generation
  • scripts/format-release.sh — release notes summary
  • Possibly scripts/compare.sh if regression output uses the same format

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions