Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add performance benchmark with hyperfine #1089

Merged
merged 2 commits into from
May 22, 2023

Conversation

tatomyr
Copy link
Contributor

@tatomyr tatomyr commented May 19, 2023

What/Why/How?

Added performance benchmarking using hyperfine.

Reference

Testing

Screenshots (optional)

Check yourself

  • Code is linted
  • Tested with redoc/reference-docs/workflows
  • All new/updated code is covered with tests

Security

  • Security impact of change has been considered
  • Code follows company security practices and guidelines

@tatomyr tatomyr force-pushed the chore/add-benchmark-action branch from 6460988 to 43c1eb7 Compare May 19, 2023 11:24
@github-actions
Copy link
Contributor

github-actions bot commented May 19, 2023

Coverage report

St.
Category Percentage Covered / Total
🟡 Statements 74.19% 3495/4711
🟡 Branches 65.04% 1981/3046
🟡 Functions 66.48% 587/883
🟡 Lines 74.24% 3253/4382

Test suite run success

585 tests passing in 85 suites.

Report generated by 🧪jest coverage report action from 35c4bbe

@tatomyr tatomyr force-pushed the chore/add-benchmark-action branch from 43c1eb7 to 955c836 Compare May 19, 2023 11:40
@github-actions
Copy link
Contributor

🚀

@tatomyr tatomyr force-pushed the chore/add-benchmark-action branch 2 times, most recently from c6c118a to aebeb3e Compare May 19, 2023 11:45
@github-actions
Copy link
Contributor

Command Mean [ms] Min [ms] Max [ms] Relative
redocly-next lint packages/core/src/benchmark/benches/rebilly.yaml 0.0 ± 0.2 0.0 5.6 1.00
redocly lint packages/core/src/benchmark/benches/rebilly.yaml 1261.0 ± 25.0 1216.8 1295.6 39048.72 ± 186672.65

@github-actions
Copy link
Contributor

🚀

@tatomyr tatomyr force-pushed the chore/add-benchmark-action branch 2 times, most recently from a724277 to 06a616a Compare May 19, 2023 11:58
@github-actions
Copy link
Contributor

Command Mean [ms] Min [ms] Max [ms] Relative
redocly lint packages/core/src/benchmark/benches/rebilly.yaml 0.0 ± 0.1 0.0 3.9 1.00
redocly-next lint packages/core/src/benchmark/benches/rebilly.yaml 1213.9 ± 18.3 1186.7 1236.5 32904.84 ± 121263.81

@tatomyr tatomyr force-pushed the chore/add-benchmark-action branch 2 times, most recently from bc14580 to bbfeed0 Compare May 19, 2023 12:54
@github-actions
Copy link
Contributor

Command Mean [s] Min [s] Max [s] Relative
redocly lint packages/core/src/benchmark/benches/rebilly.yaml 1.210 ± 0.026 1.177 1.251 1.00
redocly-next lint packages/core/src/benchmark/benches/rebilly.yaml 1.215 ± 0.024 1.179 1.244 1.00 ± 0.03

@tatomyr tatomyr force-pushed the chore/add-benchmark-action branch from bbfeed0 to b650185 Compare May 19, 2023 12:58
@github-actions
Copy link
Contributor

Command Mean [s] Min [s] Max [s] Relative
redocly lint packages/core/src/benchmark/benches/rebilly.yaml 1.056 ± 0.024 1.025 1.093 1.00
redocly-next lint packages/core/src/benchmark/benches/rebilly.yaml 1.251 ± 0.034 1.197 1.314 1.19 ± 0.04

@tatomyr tatomyr force-pushed the chore/add-benchmark-action branch from b650185 to c22f858 Compare May 19, 2023 13:05
@github-actions
Copy link
Contributor

github-actions bot commented May 19, 2023

Command Mean [s] Min [s] Max [s] Relative
redocly lint packages/core/src/benchmark/benches/rebilly.yaml 1.230 ± 0.028 1.195 1.283 1.00
redocly-next lint packages/core/src/benchmark/benches/rebilly.yaml 1.231 ± 0.032 1.183 1.273 1.00 ± 0.03

@tatomyr tatomyr force-pushed the chore/add-benchmark-action branch 3 times, most recently from c440127 to 5068589 Compare May 19, 2023 13:45
@tatomyr tatomyr force-pushed the chore/add-benchmark-action branch from 5068589 to 0cf8335 Compare May 19, 2023 13:51
@tatomyr tatomyr self-assigned this May 19, 2023
@tatomyr tatomyr marked this pull request as ready for review May 19, 2023 13:54
@tatomyr tatomyr requested a review from a team as a code owner May 19, 2023 13:54
.github/workflows/performance.yaml Outdated Show resolved Hide resolved
Comment on lines +28 to +29
- name: Install External
run: npm i -g hyperfine @redocly/cli@latest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it compare with the latest or with the version from main?

I think latest is fine for now but would be great to check-out main branch and compare it to it somehow in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, comparing to main would be better, but I haven't found a simple and reliable solution to this yet. Will check how this approach works first, and then we can decide how to evolve it.

@tatomyr tatomyr force-pushed the chore/add-benchmark-action branch from 36248c8 to 35c4bbe Compare May 20, 2023 11:35
@tatomyr tatomyr merged commit a94defe into main May 22, 2023
6 checks passed
@tatomyr tatomyr deleted the chore/add-benchmark-action branch May 22, 2023 12:31
@IgorKarpiuk IgorKarpiuk mentioned this pull request May 25, 2023
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants