Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libpython: Add helper library for benchmarking #1670

Merged
merged 3 commits into from
Jul 6, 2021

Conversation

wenzeslaus
Copy link
Member

@wenzeslaus wenzeslaus commented Jun 21, 2021

An experimental collection of simple functions to help benchmarking
and reduce code duplication between benchmarks.

The current design ideas are be minimalist, pragmatic with no API promisses.
The tools are meant for developers tracking the latest development version.

Co-authored-by: Aaron Saw Min Sern (thread/nprocs benchmarking function)

@wenzeslaus
Copy link
Member Author

@aaronsms I would be interested in what you think (I can't put you on review directly).

@wenzeslaus
Copy link
Member Author

This is addressing some of the needs from @aaronsms's GSoC 2021, but see also [GRASS-dev] measuring cpu time for each module and estimating modules scalability which was not taken into account at this point.

@wenzeslaus wenzeslaus added the enhancement New feature or request label Jun 21, 2021
@aaronsms
Copy link
Contributor

I think this is nice to add, I have tried using this for r.benchmark and r.univar. For the resolution_plot, do you think it's better to have the results be fitted into the same plot just like nprocs_plot, even if the resolutions points are slightly different, because users can always choose call resolutions_plot multiple times if they want separate graphs. Having them sharing a same graph allows us to see how it scales against bigger inputs, for example;
image

python/grass/benchmark/plots.py Show resolved Hide resolved
python/grass/benchmark/plots.py Outdated Show resolved Hide resolved
python/grass/benchmark/plots.py Show resolved Hide resolved
python/grass/benchmark/plots.py Show resolved Hide resolved
@HuidaeCho
Copy link
Member

For the resolution_plot, do you think it's better to have the results be fitted into the same plot just like nprocs_plot

I think it would be more useful to plot them all in the same plot just like nprocs_plot.

@wenzeslaus wenzeslaus marked this pull request as ready for review July 3, 2021 02:31
@wenzeslaus
Copy link
Member Author

For the resolution_plot, do you think it's better to have the results be fitted into the same plot just like nprocs_plot

I think it would be more useful to plot them all in the same plot just like nprocs_plot.

Agreed. Done.

@wenzeslaus
Copy link
Member Author

This now has documentation and test. Additional benchmarking, outputs, plots are left for future PRs.

An experimental collection of simple functions to help benchmarking
and reduce code duplication between benchmarks.

The current design ideas are be minimalist, pragmatic with no API promisses.
The tools are meant for developers tracking the latest development version.

Co-authored-by: Aaron Saw Min Sern <aaronsms>
@wenzeslaus wenzeslaus merged commit e334471 into OSGeo:master Jul 6, 2021
@wenzeslaus wenzeslaus deleted the add-benchmark-tools branch July 6, 2021 02:44
@wenzeslaus
Copy link
Member Author

I'm not sure if the test will run in the CI, but I did two rounds of dealing with the dependencies, so hopefully it will. I can't confirm that now, due to addon installation failing which fails the tests in CI.

I merged this anyway so it is available for @aaronsms and anybody who will be testing his work.

@neteler neteler added this to the 8.0.0 milestone Dec 9, 2021
ninsbl pushed a commit to ninsbl/grass that referenced this pull request Oct 26, 2022
An experimental collection of simple functions to help benchmarking
and reduce code duplication between benchmarks.

The design ideas are: be minimalist, pragmatic with no API promises,
but provide convenient functions for writing benchmarking scripts.
The functions are meant for developers tracking the latest development version.

The plotting functions can be imported with missing dependencies for convenience,
but running them requires Matplotlib.

Co-authored-by: Aaron Saw Min Sern <aaronsms@u.nus.edu>
ninsbl pushed a commit to ninsbl/grass that referenced this pull request Feb 17, 2023
An experimental collection of simple functions to help benchmarking
and reduce code duplication between benchmarks.

The design ideas are: be minimalist, pragmatic with no API promises,
but provide convenient functions for writing benchmarking scripts.
The functions are meant for developers tracking the latest development version.

The plotting functions can be imported with missing dependencies for convenience,
but running them requires Matplotlib.

Co-authored-by: Aaron Saw Min Sern <aaronsms@u.nus.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants