New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve significance image analysis #474

Merged
merged 9 commits into from Mar 1, 2016

Conversation

Projects
None yet
3 participants
@adonath
Member

adonath commented Feb 26, 2016

This PR includes a refactoring of the Li&Ma analysis in gammapy. It requires the latest version of gammapy-extra. Docs are still missing, but will be added.

@adonath adonath added this to the 0.4 milestone Feb 29, 2016

@adonath adonath self-assigned this Feb 29, 2016

@adonath

This comment has been minimized.

Show comment
Hide comment
@adonath

adonath Mar 1, 2016

Member

@cdeil This is ready for further review. I'm currently expanding the docs...but the technical stuff is ready. I'd like to merge this rather soon, beacuse I'm working on a few minor follow up PRs.

Member

adonath commented Mar 1, 2016

@cdeil This is ready for further review. I'm currently expanding the docs...but the technical stuff is ready. I'd like to merge this rather soon, beacuse I'm working on a few minor follow up PRs.

Show outdated Hide outdated gammapy/detect/lima.py Outdated
Show outdated Hide outdated gammapy/detect/lima.py Outdated
excess=n_on_ - background,
alpha=alpha)
if not exposure is None:

This comment has been minimized.

@cdeil

cdeil Mar 1, 2016

Member

if not exposure is None: -> if exposure:

@cdeil

cdeil Mar 1, 2016

Member

if not exposure is None: -> if exposure:

This comment has been minimized.

@adonath

adonath Mar 1, 2016

Member

I just checked, this doesn't work, when exposure is a numpy array. This results in ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

@adonath

adonath Mar 1, 2016

Member

I just checked, this doesn't work, when exposure is a numpy array. This results in ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Mar 1, 2016

Member

As I said recently, I don't think the current API / partitioning of functionality is good.

Putting convolution and flux computation in compute_lima_map makes for a complicated API and means the few key lines to correlate the maps and compute flux will have to be duplicated elsewhere.

But as we discussed right now ... we have bigger fish to fry ... merge and onto a joint fit for two datasets?

Member

cdeil commented Mar 1, 2016

As I said recently, I don't think the current API / partitioning of functionality is good.

Putting convolution and flux computation in compute_lima_map makes for a complicated API and means the few key lines to correlate the maps and compute flux will have to be duplicated elsewhere.

But as we discussed right now ... we have bigger fish to fry ... merge and onto a joint fit for two datasets?

Show outdated Hide outdated gammapy/data/maps.py Outdated
Show outdated Hide outdated gammapy/detect/lima.py Outdated
Show outdated Hide outdated gammapy/detect/test_statistics.py Outdated
Show outdated Hide outdated gammapy/detect/tests/test_lima.py Outdated
@adonath

This comment has been minimized.

Show comment
Hide comment
@adonath

adonath Mar 1, 2016

Member

@cdeil Thanks for the review! As I mentioned before, this doesn't necessarily represent the last final API. It just brings back broken high level functionality and allows to make other existing functionality more uniform and resolve code duplications along with that. A better, more flexible and user-friendly API can easily be later build on top, when we exactly know what it should look like.

Member

adonath commented Mar 1, 2016

@cdeil Thanks for the review! As I mentioned before, this doesn't necessarily represent the last final API. It just brings back broken high level functionality and allows to make other existing functionality more uniform and resolve code duplications along with that. A better, more flexible and user-friendly API can easily be later build on top, when we exactly know what it should look like.

cdeil added a commit that referenced this pull request Mar 1, 2016

@cdeil cdeil merged commit 1c9a56c into gammapy:master Mar 1, 2016

0 of 2 checks passed

continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
@joleroi

This comment has been minimized.

Show comment
Hide comment
@joleroi

joleroi Mar 2, 2016

Contributor

When running my gammapy-spectrum command line tool, I get

[/home/kingj/Software/gammapy-extra/test_datasets/spectrum] gammapy-spectrum extract spectrum_analysis_example.yaml 
Traceback (most recent call last):
  File "/home/kingj/Software/miniconda3/envs/headsherpa/bin/gammapy-spectrum", line 9, in <module>
    load_entry_point('gammapy', 'console_scripts', 'gammapy-spectrum')()
  File "/home/kingj/Software/miniconda3/envs/headsherpa/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg/pkg_resources/__init__.py", line 547, in load_entry_point

  File "/home/kingj/Software/miniconda3/envs/headsherpa/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg/pkg_resources/__init__.py", line 2720, in load_entry_point

  File "/home/kingj/Software/miniconda3/envs/headsherpa/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg/pkg_resources/__init__.py", line 2380, in load

  File "/home/kingj/Software/miniconda3/envs/headsherpa/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg/pkg_resources/__init__.py", line 2386, in resolve

  File "/home/kingj/Software/gammapy/gammapy/scripts/__init__.py", line 19, in <module>
    from .image_lima import *
  File "/home/kingj/Software/gammapy/gammapy/scripts/image_lima.py", line 11, in <module>
    from ..detect import compute_lima_map, compute_lima_on_off_map
  File "/home/kingj/Software/gammapy/gammapy/detect/__init__.py", line 7, in <module>
    from .test_statistics import *
  File "/home/kingj/Software/gammapy/gammapy/detect/test_statistics.py", line 22, in <module>
    from ._test_statistics_cython import (_cash_cython, _amplitude_bounds_cython,
ImportError: No module named _test_statistics_cython

This looks like it is related to this PR...

Does anyone have any idea?

Contributor

joleroi commented Mar 2, 2016

When running my gammapy-spectrum command line tool, I get

[/home/kingj/Software/gammapy-extra/test_datasets/spectrum] gammapy-spectrum extract spectrum_analysis_example.yaml 
Traceback (most recent call last):
  File "/home/kingj/Software/miniconda3/envs/headsherpa/bin/gammapy-spectrum", line 9, in <module>
    load_entry_point('gammapy', 'console_scripts', 'gammapy-spectrum')()
  File "/home/kingj/Software/miniconda3/envs/headsherpa/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg/pkg_resources/__init__.py", line 547, in load_entry_point

  File "/home/kingj/Software/miniconda3/envs/headsherpa/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg/pkg_resources/__init__.py", line 2720, in load_entry_point

  File "/home/kingj/Software/miniconda3/envs/headsherpa/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg/pkg_resources/__init__.py", line 2380, in load

  File "/home/kingj/Software/miniconda3/envs/headsherpa/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg/pkg_resources/__init__.py", line 2386, in resolve

  File "/home/kingj/Software/gammapy/gammapy/scripts/__init__.py", line 19, in <module>
    from .image_lima import *
  File "/home/kingj/Software/gammapy/gammapy/scripts/image_lima.py", line 11, in <module>
    from ..detect import compute_lima_map, compute_lima_on_off_map
  File "/home/kingj/Software/gammapy/gammapy/detect/__init__.py", line 7, in <module>
    from .test_statistics import *
  File "/home/kingj/Software/gammapy/gammapy/detect/test_statistics.py", line 22, in <module>
    from ._test_statistics_cython import (_cash_cython, _amplitude_bounds_cython,
ImportError: No module named _test_statistics_cython

This looks like it is related to this PR...

Does anyone have any idea?

@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Mar 3, 2016

Member

I don't see this issue.
Can you try make clean?

Member

cdeil commented Mar 3, 2016

I don't see this issue.
Can you try make clean?

@joleroi

This comment has been minimized.

Show comment
Hide comment
@joleroi

joleroi Mar 3, 2016

Contributor

Removing the gammapy installation in conda, and rebuilding everything fixed it

Contributor

joleroi commented Mar 3, 2016

Removing the gammapy installation in conda, and rebuilding everything fixed it

@cdeil cdeil changed the title from Refactor lima analysis to Improve significance image analysis Apr 20, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment