Skip to content

chore: Add GitHub Action to run benchmarks on pull requests#31

Merged
fengmk2 merged 4 commits intomasterfrom
copilot/add-github-action-npm-bench
Dec 3, 2025
Merged

chore: Add GitHub Action to run benchmarks on pull requests#31
fengmk2 merged 4 commits intomasterfrom
copilot/add-github-action-npm-bench

Conversation

Copy link
Contributor

Copilot AI commented Dec 3, 2025

Adds automated benchmark execution on PRs with results posted as a comment.

Changes

  • New workflow .github/workflows/benchmark.yml:
    • Triggers on PRs (excludes docs-only changes)
    • Builds with x86_64-unknown-linux-gnu target
    • Runs yarn bench and captures output
    • Posts/updates PR comment with results via peter-evans/create-or-update-comment
    • Adds results to job summary

Uses same patterns as existing workflows: pinned action SHAs, cargo caching, concurrency groups.

Original prompt

Add a GitHub action to automatically run npm run bench when a pull request is made and reply the results in the PR.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 3, 2025

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Comment @coderabbitai help to get the list of available commands and usage tips.

Co-authored-by: fengmk2 <156269+fengmk2@users.noreply.github.com>
Copilot AI changed the title [WIP] Add GitHub action to run npm bench on PR Add GitHub Action to run benchmarks on pull requests Dec 3, 2025
Copilot AI requested a review from fengmk2 December 3, 2025 12:23
@github-actions
Copy link

github-actions bot commented Dec 3, 2025

📦 napi-rs Build Size Comparison

File Base Size PR Size Diff %
packument.linux-x64-gnu.node 849.82 KB 849.82 KB 0 KB 0%

Total: 849.82 KB → 849.82 KB (0 KB, 0%)

Compared using x86_64-unknown-linux-gnu target

@github-actions
Copy link

github-actions bot commented Dec 3, 2025

⚡ Benchmark Results

Click to expand benchmark output
┌─────────┬──────────────────────────────────────────────────────────────────────┬─────────────────────┬───────────────────────┬────────────────────────┬────────────────────────┬─────────┐
│ (index) │ Task name                                                            │ Latency avg (ns)    │ Latency med (ns)      │ Throughput avg (ops/s) │ Throughput med (ops/s) │ Samples │
├─────────┼──────────────────────────────────────────────────────────────────────┼─────────────────────┼───────────────────────┼────────────────────────┼────────────────────────┼─────────┤
│ 0       │ 'JSONParse small data readme string (117KB)'                         │ '542426 ± 0.74%'    │ '499854 ± 13931'      │ '1879 ± 0.57%'         │ '2001 ± 57'            │ 1844    │
│ 1       │ 'sonic-rs small data readme string (117KB)'                          │ '125438 ± 0.25%'    │ '121326 ± 521.00'     │ '8037 ± 0.16%'         │ '8242 ± 36'            │ 7973    │
│ 2       │ 'sonic-rs small data readme string with position (117KB)'            │ '146102 ± 0.31%'    │ '140983 ± 872.00'     │ '6906 ± 0.18%'         │ '7093 ± 44'            │ 6845    │
│ 3       │ 'sonic-rs small data readme JSON buffer with position (117KB)'       │ '104775 ± 0.20%'    │ '103363 ± 131.00'     │ '9571 ± 0.07%'         │ '9675 ± 12'            │ 9545    │
│ 4       │ 'JSONParse large data readme string (22MB)'                          │ '113370962 ± 1.50%' │ '112090777 ± 445246'  │ '9 ± 1.13%'            │ '9 ± 0'                │ 64      │
│ 5       │ 'sonic-rs large data readme string (22MB)'                           │ '18162316 ± 0.17%'  │ '18140690 ± 36077'    │ '55 ± 0.17%'           │ '55 ± 0'               │ 64      │
│ 6       │ 'sonic-rs large data readme string with position (22MB)'             │ '18163143 ± 0.19%'  │ '18122271 ± 36729'    │ '55 ± 0.19%'           │ '55 ± 0'               │ 64      │
│ 7       │ 'sonic-rs large data readme JSON buffer with position (22MB)'        │ '18131005 ± 0.08%'  │ '18122110 ± 31349'    │ '55 ± 0.08%'           │ '55 ± 0'               │ 64      │
│ 8       │ 'JSONParse super large data readme string (89M)'                     │ '388394284 ± 1.08%' │ '377564073 ± 2609466' │ '3 ± 1.05%'            │ '3 ± 0'                │ 64      │
│ 9       │ 'sonic-rs super large data readme string (89M)'                      │ '66274727 ± 0.07%'  │ '66243400 ± 105205'   │ '15 ± 0.07%'           │ '15 ± 0'               │ 64      │
│ 10      │ 'sonic-rs super large data readme string with position (89M)'        │ '66201439 ± 0.06%'  │ '66209247 ± 105682'   │ '15 ± 0.06%'           │ '15 ± 0'               │ 64      │
│ 11      │ 'sonic-rs super large data readme JSON buffer with position (89M)'   │ '66197156 ± 0.09%'  │ '66189530 ± 107500'   │ '15 ± 0.09%'           │ '15 ± 0'               │ 64      │
│ 12      │ 'JSONParse big readme string (229KB, 64KB readme)'                   │ '734827 ± 3.14%'    │ '697306 ± 10240'      │ '1417 ± 0.40%'         │ '1434 ± 21'            │ 1361    │
│ 13      │ 'sonic-rs big readme string (229KB, 64KB readme)'                    │ '261721 ± 0.34%'    │ '258190 ± 1923.0'     │ '3833 ± 0.11%'         │ '3873 ± 29'            │ 3821    │
│ 14      │ 'sonic-rs big readme string with position (229KB, 64KB readme)'      │ '275679 ± 0.24%'    │ '271104 ± 1793.0'     │ '3638 ± 0.14%'         │ '3689 ± 25'            │ 3628    │
│ 15      │ 'sonic-rs big readme JSON buffer with position (229KB, 64KB readme)' │ '209095 ± 0.15%'    │ '205922 ± 402.00'     │ '4792 ± 0.11%'         │ '4856 ± 9'             │ 4783    │
│ 16      │ 'JSONParse large data add version (22MB)'                            │ '272187503 ± 0.11%' │ '272093741 ± 534446'  │ '4 ± 0.11%'            │ '4 ± 0'                │ 64      │
│ 17      │ 'sonic-rs large data add version (22MB)'                             │ '49865277 ± 0.62%'  │ '49201496 ± 659266'   │ '20 ± 0.61%'           │ '20 ± 0'               │ 64      │
│ 18      │ 'JSONParse super large data add version (89M)'                       │ '797123416 ± 0.12%' │ '797244377 ± 1458011' │ '1 ± 0.13%'            │ '1 ± 0'                │ 64      │
│ 19      │ 'sonic-rs super large data add version (89M)'                        │ '193793067 ± 0.19%' │ '193359623 ± 614877'  │ '5 ± 0.19%'            │ '5 ± 0'                │ 64      │
└─────────┴──────────────────────────────────────────────────────────────────────┴─────────────────────┴───────────────────────┴────────────────────────┴────────────────────────┴─────────┘

Memory Usage:
  JSONParse description (22M): 472.4 MB (min: 471.8 MB, max: 473.4 MB)
  JSONParse description (89M): 764.9 MB (min: 764.0 MB, max: 766.2 MB)
  SonicJSONParse description (22M): 91.0 MB (min: 89.9 MB, max: 91.9 MB)
  SonicJSONParse description (89M): 158.3 MB (min: 157.6 MB, max: 159.4 MB)

Memory Usage:
  JSONParse add version (22M): 419.8 MB (min: 419.2 MB, max: 420.8 MB)
  JSONParse add version (89M): 778.3 MB (min: 777.3 MB, max: 779.4 MB)
  SonicJSONParse add version (22M): 183.1 MB (min: 181.2 MB, max: 183.8 MB)
  SonicJSONParse add version (89M): 426.7 MB (min: 425.9 MB, max: 427.7 MB)

## Benchmarking JSONParse with @primer/react.json
- Data size: 89MB
- GC: enabled

JSONParse add version (@primer/react.json) #1 time: 862ms
JSONParse add version (@primer/react.json) #2 time: 793ms
JSONParse add version (@primer/react.json) #3 time: 836ms
JSONParse add version (@primer/react.json) #4 time: 804ms
JSONParse add version (@primer/react.json) #5 time: 798ms

[GC] total(ms)= 134.02 count= 24 avg(ms)= 5.58 byKind= {
  scavenge: 55.939808999886736,
  markSweepCompact: 76.35713800002122,
  incremental: 1.7214430000749417,
  weakc: 0,
  unknown: 0
}

## Benchmarking SonicJSONParse with @primer/react.json
- Data size: 89MB
- GC: enabled

SonicJSONParse add version (@primer/react.json) #1 time: 195ms
SonicJSONParse add version (@primer/react.json) #2 time: 194ms
SonicJSONParse add version (@primer/react.json) #3 time: 193ms
SonicJSONParse add version (@primer/react.json) #4 time: 194ms
SonicJSONParse add version (@primer/react.json) #5 time: 193ms

[GC] total(ms)= 5.15 count= 8 avg(ms)= 0.64 byKind= {
  scavenge: 0,
  markSweepCompact: 4.298681999964174,
  incremental: 0.8544380000093952,
  weakc: 0,
  unknown: 0
}

Benchmark ran on x86_64-unknown-linux-gnu target

@fengmk2 fengmk2 changed the title Add GitHub Action to run benchmarks on pull requests chore: Add GitHub Action to run benchmarks on pull requests Dec 3, 2025
@fengmk2 fengmk2 marked this pull request as ready for review December 3, 2025 12:30
Copilot AI review requested due to automatic review settings December 3, 2025 12:30
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds automated benchmark execution on pull requests, following the established patterns from existing workflows. The implementation posts benchmark results as PR comments and includes them in the job summary.

  • Adds .github/workflows/benchmark.yml that triggers on PRs (excluding documentation changes)
  • Builds the project for the x86_64-unknown-linux-gnu target and runs yarn bench
  • Posts/updates a collapsible PR comment with benchmark results using the peter-evans actions

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@fengmk2 fengmk2 merged commit 425ce00 into master Dec 3, 2025
38 checks passed
@fengmk2 fengmk2 deleted the copilot/add-github-action-npm-bench branch December 3, 2025 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants