Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use precomputed mean/std to compute skewness and kurtosis #1349

Merged
merged 7 commits into from
May 20, 2020

Conversation

maxnoe
Copy link
Member

@maxnoe maxnoe commented May 20, 2020

master:

334 µs ± 1.48 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

this:

55.7 µs ± 549 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

ctapipe/image/statistics.py Outdated Show resolved Hide resolved
ctapipe/image/statistics.py Outdated Show resolved Hide resolved
dneise
dneise previously approved these changes May 20, 2020
Copy link
Member

@dneise dneise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it

@watsonjj
Copy link
Contributor

watsonjj commented May 20, 2020

These are simple mathematical operations and has couple of temporary arrays....you could check what further performance you get with numba's njit, and maybe with (fastmath=True)

@maxnoe
Copy link
Member Author

maxnoe commented May 20, 2020

I tried with njit gets us another 10 %.

@codecov
Copy link

codecov bot commented May 20, 2020

Codecov Report

Merging #1349 into master will decrease coverage by 0.06%.
The diff coverage is 88.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1349      +/-   ##
==========================================
- Coverage   91.10%   91.04%   -0.07%     
==========================================
  Files         183      184       +1     
  Lines       12548    12609      +61     
==========================================
+ Hits        11432    11480      +48     
- Misses       1116     1129      +13     
Impacted Files Coverage Δ
ctapipe/__init__.py 100.00% <ø> (ø)
ctapipe/calib/camera/calibrator.py 100.00% <ø> (ø)
ctapipe/calib/camera/gainselection.py 96.55% <0.00%> (ø)
...tapipe/coordinates/tests/test_engineering_frame.py 100.00% <ø> (ø)
ctapipe/coordinates/tests/test_roundtrip.py 100.00% <ø> (ø)
ctapipe/core/__init__.py 100.00% <ø> (ø)
ctapipe/core/logging.py 100.00% <ø> (ø)
ctapipe/core/support.py 100.00% <ø> (ø)
ctapipe/core/tool.py 89.93% <ø> (ø)
ctapipe/image/__init__.py 100.00% <ø> (ø)
... and 247 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 23bf356...aed86a1. Read the comment docs.

watsonjj
watsonjj previously approved these changes May 20, 2020
@maxnoe maxnoe dismissed stale reviews from watsonjj and dneise via 7a5f1b1 May 20, 2020 14:07
@maxnoe
Copy link
Member Author

maxnoe commented May 20, 2020

Added more tests and njit, since it gave a 10 % improvement. Pls. review again.

@maxnoe maxnoe requested review from dneise and watsonjj May 20, 2020 14:08
Copy link
Contributor

@kosack kosack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you run "black", or better yet try "pre-commit install" and see if it works properly ("pre-commit run" to manually run it). Mainly just to be pep 257-complant:

"For consistency, always use """triple double quotes""" around docstrings."

Otherwise looks fine! Nice to get a bit more speed without much effort.

@maxnoe
Copy link
Member Author

maxnoe commented May 20, 2020

@kosack done

kosack
kosack previously approved these changes May 20, 2020
watsonjj
watsonjj previously approved these changes May 20, 2020
@maxnoe maxnoe dismissed stale reviews from watsonjj and kosack via aed86a1 May 20, 2020 15:22
@maxnoe
Copy link
Member Author

maxnoe commented May 20, 2020

pytest cov seems to be unable to work with the njit functions. This is unfortunate but also not really surprising in hindsight.

This is way diff coverage is so low

@kosack kosack merged commit b7af71b into master May 20, 2020
@maxnoe maxnoe deleted the improve_stats_performance branch May 20, 2020 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants