-
Notifications
You must be signed in to change notification settings - Fork 283
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove "on-demand" from some benchmarks #5959
Conversation
…e minimum memory threshold.
CPerf and SPerf would be inappropriate for other reasons - CPerf relies on local files, and SPerf can take a long time due to deliberately large data. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please could we get a What's New entry. The Internal
section is beginning to serve as a good account of where we've been busy recently.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #5959 +/- ##
=======================================
Coverage 89.78% 89.78%
=======================================
Files 93 93
Lines 23007 23007
Branches 5017 5017
=======================================
Hits 20657 20657
Misses 1620 1620
Partials 730 730 ☔ View full report in Codecov by Sentry. |
⏱️ Performance Benchmark Report: 556c9b8Performance shifts
Full benchmark results
Generated by GHA run |
Oh, I still missed the whatsnew. |
See #5961 |
* main: (759 commits) Bump scitools/workflows from 2024.05.1 to 2024.06.0 (SciTools#5986) [pre-commit.ci] pre-commit autoupdate (SciTools#5980) Updated environment lockfiles (SciTools#5983) Bump scitools/workflows from 2024.05.0 to 2024.05.1 (SciTools#5984) Make `slices_over` tests go faster (SciTools#5973) Updated environment lockfiles (SciTools#5979) Update lock files with associated fixes (SciTools#5953) List 25 slowest tests (SciTools#5969) used a note to highlight some text (SciTools#5971) Lazy `iris.cube.Cube.rolling_window` (SciTools#5795) Add memory benchmarks (SciTools#5960) Whatsnew for several benchmark developments. (SciTools#5961) Remove "on-demand" from some benchmarks (SciTools#5959) Add bm_runner 'trialrun' subcommand. (SciTools#5957) Automatically install iris-test-data for benchmark data generation (SciTools#5958) Added benchmarks for collapse and aggregate (SciTools#5954) Use tracemalloc for memory measurements. (SciTools#5948) Provide a Nox `benchmarks` session as the recommended entry point (SciTools#5951) [pre-commit.ci] pre-commit autoupdate (SciTools#5952) Remove unit benchmarks (SciTools#5949) ...
Closes #5950
Since we believe the new tracemalloc memory measurement method (TMMM ™️) ought to deliver more stable results,
Let's see what happens if we put those back in the standard set.
There are a lot more in cperf/sperf, but I think we decided to remove all of those from the standard set.
I'd be open to reconsidering that -- if anyone thinks appropriate, @trexfeathers ?