Skip to content

Caches for continuous benchmarking should be branch specific#805

Merged
lorenzwalthert merged 1 commit intor-lib:masterfrom
lorenzwalthert:fix-cb
May 26, 2021
Merged

Caches for continuous benchmarking should be branch specific#805
lorenzwalthert merged 1 commit intor-lib:masterfrom
lorenzwalthert:fix-cb

Conversation

@lorenzwalthert
Copy link
Collaborator

@lorenzwalthert lorenzwalthert commented May 25, 2021

Otherwise, cache_applying takes long the first time to run (as the cache has to be built), and then is instant, but when you compute the average per branch, one will have the first run which will drastically increase the mean.

@codecov-commenter
Copy link

codecov-commenter commented May 25, 2021

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.46%. Comparing base (014d1c8) to head (04a4370).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #805   +/-   ##
=======================================
  Coverage   90.46%   90.46%           
=======================================
  Files          47       47           
  Lines        2433     2433           
=======================================
  Hits         2201     2201           
  Misses        232      232           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confience interval in relative change) if 4e2d570 is merged into master:

  • cache_applying: 0.05s -> 0.05s [-0.85%, +4.62%]
  • cache_recording: 1.07s -> 1.07s [-1.33%, +1.73%]
  • without_cache: 2.85s -> 2.86s [-0.94%, +1.39%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@lorenzwalthert lorenzwalthert merged commit bdf5716 into r-lib:master May 26, 2021
@lorenzwalthert lorenzwalthert deleted the fix-cb branch May 26, 2021 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants