Skip to content

example PR introducing performance regression #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

mdjastrzebski
Copy link
Member

Summary

This PR intentionally introduces performance regression to showcase how Reassure detects and present such case.

Test plan

@mdjastrzebski mdjastrzebski force-pushed the feat/regression-example branch 3 times, most recently from 4d87a25 to fc93e6a Compare January 31, 2025 13:11
@callstack callstack deleted a comment from github-actions bot Jan 31, 2025
Copy link

github-actions bot commented Jan 31, 2025

Performance Comparison Report

  • Current: f335d53 - 2025-06-05 15:47:52Z
  • Baseline: main (65f2227) - 2025-06-05 15:47:31Z

Significant Changes To Duration

Show entries
Name Type Duration Count
Web Vite - AsyncComponent (50 runs) render 60.9 ms → 66.3 ms (+5.4 ms, +8.8%) 🔴 4 → 5 (+1, +25.0%) 🔴
Web Vite - TestList (100 items) render 14.8 ms → 19.5 ms (+4.7 ms, +31.8%) 🔴 1 → 2 (+1, +100.0%) 🔴
Show details
Name Type Duration Count
Web Vite - AsyncComponent (50 runs) render Baseline
Mean: 60.9 ms
Stdev: 2.5 ms (4.1%)
Runs: 60.5 60.8 60.3 61.8 61.3 61.3 58.6 52.0 61.1 61.2 61.7 61.0 61.4 61.2 61.1 62.0 61.2 61.5 52.5 61.5 61.4 61.0 61.4 66.1 63.9 62.8 60.9 61.4 60.8 61.1 60.6 64.1 63.3 62.1 58.6 60.6
Warmup runs: 61.4
Removed outliers: 37.5 41.5 33.3 45.6 34.0 37.8 45.8 33.5 37.5 39.5 35.9 37.3 39.1 33.2

Current
Mean: 66.3 ms
Stdev: 10.0 ms (15.1%)
Runs: 71.6 71.4 42.5 72.1 72.1 71.8 58.5 72.1 72.1 73.8 58.8 58.0 72.2 78.3 69.1 52.5 71.8 72.2 72.4 43.1 72.0 71.8 72.1 58.2 72.2 73.2 72.7 61.8 41.0 63.9 72.2 72.8 65.9 51.6 72.4 73.9 72.1 43.4 71.8 72.2 72.5 58.6
Warmup runs: 72.5
Removed outliers: 38.5 38.7 38.5 38.4 38.4 38.5 38.8 38.7
Baseline
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
Render issues:

Current
Mean: 5
Stdev: 0 (0.0%)
Runs: 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
Render issues:
- Initial updates: 1
Web Vite - TestList (100 items) render Baseline
Mean: 14.8 ms
Stdev: 0.6 ms (4.3%)
Runs: 14.8 16.5 14.6 14.8 14.5 14.7 14.6 14.3 14.3 14.5
Warmup runs: 14.7
Removed outliers: (none)

Current
Mean: 19.5 ms
Stdev: 0.2 ms (1.0%)
Runs: 19.7 19.9 19.4 19.4 19.5 19.5 19.4 19.3 19.4 19.1
Warmup runs: 20.0
Removed outliers: (none)
Baseline
Mean: 1
Stdev: 0 (0.0%)
Runs: 1 1 1 1 1 1 1 1 1 1
Render issues:

Current
Mean: 2
Stdev: 0 (0.0%)
Runs: 2 2 2 2 2 2 2 2 2 2
Render issues:
- Initial updates: 1

Meaningless Changes To Duration

Show entries
Name Type Duration Count
Web Vite - AsyncComponent (10 runs) render 52.1 ms → 58.9 ms (+6.8 ms, +13.0%) 4 → 5 (+1, +25.0%) 🔴
Show details
Name Type Duration Count
Web Vite - AsyncComponent (10 runs) render Baseline
Mean: 52.1 ms
Stdev: 10.0 ms (19.3%)
Runs: 55.5 56.7 58.7 37.6 45.4 60.7 60.4 61.2 51.4 33.4
Warmup runs: 66.3
Removed outliers: (none)

Current
Mean: 58.9 ms
Stdev: 12.2 ms (20.7%)
Runs: 65.4 69.2 56.0 38.9 63.0 70.9 71.1 63.5 39.0 51.8
Warmup runs: 75.0
Removed outliers: (none)
Baseline
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4
Render issues:

Current
Mean: 5
Stdev: 0 (0.0%)
Runs: 5 5 5 5 5 5 5 5 5 5
Render issues:
- Initial updates: 1

Render Count Changes

Show entries
Name Type Duration Count
Web Vite - AsyncComponent (10 runs) render 52.1 ms → 58.9 ms (+6.8 ms, +13.0%) 4 → 5 (+1, +25.0%) 🔴
Web Vite - AsyncComponent (50 runs) render 60.9 ms → 66.3 ms (+5.4 ms, +8.8%) 🔴 4 → 5 (+1, +25.0%) 🔴
Web Vite - TestList (100 items) render 14.8 ms → 19.5 ms (+4.7 ms, +31.8%) 🔴 1 → 2 (+1, +100.0%) 🔴
Show details
Name Type Duration Count
Web Vite - AsyncComponent (10 runs) render Baseline
Mean: 52.1 ms
Stdev: 10.0 ms (19.3%)
Runs: 55.5 56.7 58.7 37.6 45.4 60.7 60.4 61.2 51.4 33.4
Warmup runs: 66.3
Removed outliers: (none)

Current
Mean: 58.9 ms
Stdev: 12.2 ms (20.7%)
Runs: 65.4 69.2 56.0 38.9 63.0 70.9 71.1 63.5 39.0 51.8
Warmup runs: 75.0
Removed outliers: (none)
Baseline
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4
Render issues:

Current
Mean: 5
Stdev: 0 (0.0%)
Runs: 5 5 5 5 5 5 5 5 5 5
Render issues:
- Initial updates: 1
Web Vite - AsyncComponent (50 runs) render Baseline
Mean: 60.9 ms
Stdev: 2.5 ms (4.1%)
Runs: 60.5 60.8 60.3 61.8 61.3 61.3 58.6 52.0 61.1 61.2 61.7 61.0 61.4 61.2 61.1 62.0 61.2 61.5 52.5 61.5 61.4 61.0 61.4 66.1 63.9 62.8 60.9 61.4 60.8 61.1 60.6 64.1 63.3 62.1 58.6 60.6
Warmup runs: 61.4
Removed outliers: 37.5 41.5 33.3 45.6 34.0 37.8 45.8 33.5 37.5 39.5 35.9 37.3 39.1 33.2

Current
Mean: 66.3 ms
Stdev: 10.0 ms (15.1%)
Runs: 71.6 71.4 42.5 72.1 72.1 71.8 58.5 72.1 72.1 73.8 58.8 58.0 72.2 78.3 69.1 52.5 71.8 72.2 72.4 43.1 72.0 71.8 72.1 58.2 72.2 73.2 72.7 61.8 41.0 63.9 72.2 72.8 65.9 51.6 72.4 73.9 72.1 43.4 71.8 72.2 72.5 58.6
Warmup runs: 72.5
Removed outliers: 38.5 38.7 38.5 38.4 38.4 38.5 38.8 38.7
Baseline
Mean: 4
Stdev: 0 (0.0%)
Runs: 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
Render issues:

Current
Mean: 5
Stdev: 0 (0.0%)
Runs: 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
Render issues:
- Initial updates: 1
Web Vite - TestList (100 items) render Baseline
Mean: 14.8 ms
Stdev: 0.6 ms (4.3%)
Runs: 14.8 16.5 14.6 14.8 14.5 14.7 14.6 14.3 14.3 14.5
Warmup runs: 14.7
Removed outliers: (none)

Current
Mean: 19.5 ms
Stdev: 0.2 ms (1.0%)
Runs: 19.7 19.9 19.4 19.4 19.5 19.5 19.4 19.3 19.4 19.1
Warmup runs: 20.0
Removed outliers: (none)
Baseline
Mean: 1
Stdev: 0 (0.0%)
Runs: 1 1 1 1 1 1 1 1 1 1
Render issues:

Current
Mean: 2
Stdev: 0 (0.0%)
Runs: 2 2 2 2 2 2 2 2 2 2
Render issues:
- Initial updates: 1

Render Issues

Name Initial Updates Redundant Updates
Web Vite - AsyncComponent (10 runs) 1 🔴 -
Web Vite - AsyncComponent (50 runs) 1 🔴 -
Web Vite - TestList (100 items) 1 🔴 -

Added Entries

There are no entries

Removed Entries

There are no entries

Generated by 🚫 dangerJS against 416970d

@Victorvikson1996
Copy link

How can Use this

@mdjastrzebski
Copy link
Member Author

@Victorvikson1996 you can learn more about Reassure here: docs, GitHub.

@Victorvikson1996
Copy link

Thank you so much @mdjastrzebski

@fabioh8010 fabioh8010 force-pushed the feat/regression-example branch from fc93e6a to 416970d Compare June 5, 2025 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants