New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve npred cube functionality #151

Merged
merged 7 commits into from Aug 20, 2014

Conversation

Projects
None yet
2 participants
@ellisowen
Contributor

ellisowen commented Jul 24, 2014

Implementation which I intend to use to produce significance images (currently this is still running on my machine).

The unit tests will need much improvement soon as I just implemented some quick ones. Looking at the outputs with ds9 using the FermiGalacticCenter files seemed reasonable, but obviously not rigorous yet.

Don't know if you were aiming to merge this asap, so did a pull request just in case.

@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Jul 24, 2014

Member

No need to merge this ASAP.
I won't have time to trying it out today, so just continue on your machine for now (and add tests that show that this is doing what it's supposed to do) and I'll review this on Monday.

Member

cdeil commented Jul 24, 2014

No need to merge this ASAP.
I won't have time to trying it out today, so just continue on your machine for now (and add tests that show that this is doing what it's supposed to do) and I'll review this on Monday.

@ellisowen

This comment has been minimized.

Show comment
Hide comment
@ellisowen

ellisowen Jul 25, 2014

Contributor

To do (Ellis):

  • Bug fix
  • Include documents of tests against gtmodel for Vela region
Contributor

ellisowen commented Jul 25, 2014

To do (Ellis):

  • Bug fix
  • Include documents of tests against gtmodel for Vela region

@ellisowen ellisowen referenced this pull request Jul 26, 2014

Merged

Add Fermi Vela dataset #156

@ellisowen

This comment has been minimized.

Show comment
Hide comment
@ellisowen

ellisowen Jul 29, 2014

Contributor

Pushed for discussion & review... I think this will need PR #156 operational to work

Contributor

ellisowen commented Jul 29, 2014

Pushed for discussion & review... I think this will need PR #156 operational to work

Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Jul 30, 2014

Member

I can review this tonight.

There's a few TODO left in the code that should be resolved before this can be merged.

And for your example scripts, please add an empty line every once in a while where a new step starts (e.g. compute A, compute B, plot C) ... I find it hard to get a quick overview of what this code does if it's so dense.

Member

cdeil commented Jul 30, 2014

I can review this tonight.

There's a few TODO left in the code that should be resolved before this can be merged.

And for your example scripts, please add an empty line every once in a while where a new step starts (e.g. compute A, compute B, plot C) ... I find it hard to get a quick overview of what this code does if it's so dense.

Show outdated Hide outdated docs/tutorials/npred_convolved.py Outdated
Show outdated Hide outdated docs/tutorials/npred_convolved.py Outdated
@ellisowen

This comment has been minimized.

Show comment
Hide comment
@ellisowen

ellisowen Jul 31, 2014

Contributor

I think this addresses your comments.

The Travis CI failure is due to two time out errors, so I guess I don't need to worry about this?

Contributor

ellisowen commented Jul 31, 2014

I think this addresses your comments.

The Travis CI failure is due to two time out errors, so I guess I don't need to worry about this?

@@ -140,32 +140,76 @@ def test_spatial_coordinate_images(self):
# TODO assert the four corner values
@pytest.mark.skipif('not HAS_SCIPY')

This comment has been minimized.

@cdeil

cdeil Aug 4, 2014

Member

reproject is an optional dependency ... please add a skipif to any tests that use it.
Note that you can "stack" decorators, i.e. simply have two lines:

@pytest.mark.skipif('not HAS_SCIPY')
@pytest.mark.skipif('not HAS_REPROJECT')
@cdeil

cdeil Aug 4, 2014

Member

reproject is an optional dependency ... please add a skipif to any tests that use it.
Note that you can "stack" decorators, i.e. simply have two lines:

@pytest.mark.skipif('not HAS_SCIPY')
@pytest.mark.skipif('not HAS_REPROJECT')

This comment has been minimized.

@ellisowen

ellisowen Aug 6, 2014

Contributor

I changed this in the line below so it didn't spot the change in the diff here

@ellisowen

ellisowen Aug 6, 2014

Contributor

I changed this in the line below so it didn't spot the change in the diff here

Show outdated Hide outdated docs/tutorials/npred_convolved.py Outdated
@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Aug 4, 2014

Member

Can you please move the tutorial in a sub-folder docs/tutorials/npred?
(we really should do this with all tutorials ... as this grows it will otherwise be hard to know which .rst or .py files belong together.

Member

cdeil commented Aug 4, 2014

Can you please move the tutorial in a sub-folder docs/tutorials/npred?
(we really should do this with all tutorials ... as this grows it will otherwise be hard to know which .rst or .py files belong together.

Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
Show outdated Hide outdated docs/tutorials/predicted_counts.rst Outdated

@cdeil cdeil referenced this pull request Aug 4, 2014

Merged

Add code to make model images from a source catalog #160

5 of 5 tasks complete
Show outdated Hide outdated docs/tutorials/npred_convolved.py Outdated
Show outdated Hide outdated docs/tutorials/npred_convolved.py Outdated
Show outdated Hide outdated docs/tutorials/npred_convolved.py Outdated
Show outdated Hide outdated docs/tutorials/npred_convolved.py Outdated
Show outdated Hide outdated docs/tutorials/npred_convolved.py Outdated
Show outdated Hide outdated docs/tutorials/predicted_counts.rst Outdated
Show outdated Hide outdated docs/tutorials/predicted_counts.rst Outdated
@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Aug 5, 2014

Member

@ellisowen - I've merged the other related PRs ... can you update this one tomorrow?

Member

cdeil commented Aug 5, 2014

@ellisowen - I've merged the other related PRs ... can you update this one tomorrow?

@ellisowen

This comment has been minimized.

Show comment
Hide comment
@ellisowen

ellisowen Aug 5, 2014

Contributor

Yes, should be updated by first thing tomorrow

Contributor

ellisowen commented Aug 5, 2014

Yes, should be updated by first thing tomorrow

@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Aug 5, 2014

Member

This might need some more review ... I couldn't try this out yet because my internet connection is super-bad and the 500 MB diffuse model file download always failed.
So if you have a working version where the examples in the Sphinx docs build work with the small file, please push to Github and let me know.

Member

cdeil commented Aug 5, 2014

This might need some more review ... I couldn't try this out yet because my internet connection is super-bad and the 500 MB diffuse model file download always failed.
So if you have a working version where the examples in the Sphinx docs build work with the small file, please push to Github and let me know.

desired = 1.0
energy_band = Quantity([10, 500], 'GeV')
psf_band = psf.table_psf_in_energy_band(energy_band)
actual = psf_band.kernel(pixel_size, pixel_size, normalize=True).value.sum()

This comment has been minimized.

@cdeil

cdeil Aug 18, 2014

Member

The docstring for the kernel method says an array is returned.
Apparently currently a Quantity is returned instead, or why do you call .value here?
Please make this consistent (I think returning an array and not a quantity would be OK here).

@cdeil

cdeil Aug 18, 2014

Member

The docstring for the kernel method says an array is returned.
Apparently currently a Quantity is returned instead, or why do you call .value here?
Please make this consistent (I think returning an array and not a quantity would be OK here).

This comment has been minimized.

@ellisowen

ellisowen Aug 18, 2014

Contributor

I have changed the docstring to say it returns a Quantity so at least it's consistent - I didn't write this particular function, so I don't really want to make changes to it (I imagine switching to returning an array could lead to needing to make changes elsewhere as the function is used in a number of places in gammapy, and this could end up being quite a bit of work...)

@ellisowen

ellisowen Aug 18, 2014

Contributor

I have changed the docstring to say it returns a Quantity so at least it's consistent - I didn't write this particular function, so I don't really want to make changes to it (I imagine switching to returning an array could lead to needing to make changes elsewhere as the function is used in a number of places in gammapy, and this could end up being quite a bit of work...)

Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Aug 18, 2014

Member

It's great that you now have consistent npred between your code and gtmodel!

I've left a few minor inline comments, but this is basically ready once you fix the issue mentioned at gammapy/gammapy-extra#6 (comment) .

This will change some of the unit tests reference values, so please make sure to run python setup.py test --remote-data and python setup.py build_sphinx to make sure this is OK.

About the model images you show above ... can you change the color scale for the model images (not the ratio image)? It's all blue now and I can't see anything.
You might also want to use projection_type ='bicubic' ... that should give smoother and better results (and be a bit slower, but that doesn't matter here).

Member

cdeil commented Aug 18, 2014

It's great that you now have consistent npred between your code and gtmodel!

I've left a few minor inline comments, but this is basically ready once you fix the issue mentioned at gammapy/gammapy-extra#6 (comment) .

This will change some of the unit tests reference values, so please make sure to run python setup.py test --remote-data and python setup.py build_sphinx to make sure this is OK.

About the model images you show above ... can you change the color scale for the model images (not the ratio image)? It's all blue now and I can't see anything.
You might also want to use projection_type ='bicubic' ... that should give smoother and better results (and be a bit slower, but that doesn't matter here).

@ellisowen

This comment has been minimized.

Show comment
Hide comment
@ellisowen

ellisowen Aug 18, 2014

Contributor

Some of the Travis CI failures appear to relate to stuff that isn't changed in this PR, the rest should pass when PR 6 in gammapy-extra (vela corrections) is merged.

Otherwise, is this OK?

Contributor

ellisowen commented Aug 18, 2014

Some of the Travis CI failures appear to relate to stuff that isn't changed in this PR, the rest should pass when PR 6 in gammapy-extra (vela corrections) is merged.

Otherwise, is this OK?

@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Aug 19, 2014

Member

Restarted travis-ci after the other pull request has been merged and the unrelated error has been fixed ... let's see.

Member

cdeil commented Aug 19, 2014

Restarted travis-ci after the other pull request has been merged and the unrelated error has been fixed ... let's see.

Show outdated Hide outdated gammapy/datasets/load.py Outdated
Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
Show outdated Hide outdated gammapy/spectral_cube/core.py Outdated
return hdu_list
def writeto(self, filename, clobber=False):

This comment has been minimized.

@cdeil

cdeil Aug 19, 2014

Member

I still think this method is superfluous, because

cube.to_fits().writeto('cube.fits')

is almost as short as

cube.writeto('cube.fits')

But if you absolutely want to keep it, please implement it as the one line

self.to_fits.writeto(filename, clobber)

instead of re-writing the three lines from to_fits.

@cdeil

cdeil Aug 19, 2014

Member

I still think this method is superfluous, because

cube.to_fits().writeto('cube.fits')

is almost as short as

cube.writeto('cube.fits')

But if you absolutely want to keep it, please implement it as the one line

self.to_fits.writeto(filename, clobber)

instead of re-writing the three lines from to_fits.

@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Aug 19, 2014

Member

OK ... finally I was able to build the Sphinx docs for this ... I've put them here for reference:
http://www.mpi-hd.mpg.de/personalhomes/deil/public/gammapy/issue_151/tutorials/npred/predicted_counts.html

  • I thought now the predicted counts agreed completely ... now I see again gtmodel_npred = 265 and gammapy_npred = 281. Are these simply outdated numbers or are results non-consistent again?
  • I don't understand the content of any of the images shown. The background model should look like what ds9 gammapy-extra/datasets/vela_region/background_vela.fits shows like, no? (see attached screenshot). Using the same color scale for the npred images and ratio image is confusing ... I'd suggest plotting the ratio image separately with it's own colorbar.
  • The gtmodel image looks smoother ... does the PSF convolution work properly?
  • The significance images don't show Vela? In any case I'd suggest removing them from this tutorial or putting it at the end, because the npred computation is a separate first step and the computation of a significance image is a separate later step.
  • Don't repeat the dataset parameters (the bullet list) here ... this is hard to maintain ... instead link to this README file.

screen shot 2014-08-19 at 14 20 03

Member

cdeil commented Aug 19, 2014

OK ... finally I was able to build the Sphinx docs for this ... I've put them here for reference:
http://www.mpi-hd.mpg.de/personalhomes/deil/public/gammapy/issue_151/tutorials/npred/predicted_counts.html

  • I thought now the predicted counts agreed completely ... now I see again gtmodel_npred = 265 and gammapy_npred = 281. Are these simply outdated numbers or are results non-consistent again?
  • I don't understand the content of any of the images shown. The background model should look like what ds9 gammapy-extra/datasets/vela_region/background_vela.fits shows like, no? (see attached screenshot). Using the same color scale for the npred images and ratio image is confusing ... I'd suggest plotting the ratio image separately with it's own colorbar.
  • The gtmodel image looks smoother ... does the PSF convolution work properly?
  • The significance images don't show Vela? In any case I'd suggest removing them from this tutorial or putting it at the end, because the npred computation is a separate first step and the computation of a significance image is a separate later step.
  • Don't repeat the dataset parameters (the bullet list) here ... this is hard to maintain ... instead link to this README file.

screen shot 2014-08-19 at 14 20 03

@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Aug 19, 2014

Member

I know we've discussed this before and there was some problem, but for sky images in the docs we really should use aplpy.FITSFigure.
It should just work if you pass it a filename or an HDU.
Could you please give this another try and paste the exact code / error message here if you can't get this to work?

Member

cdeil commented Aug 19, 2014

I know we've discussed this before and there was some problem, but for sky images in the docs we really should use aplpy.FITSFigure.
It should just work if you pass it a filename or an HDU.
Could you please give this another try and paste the exact code / error message here if you can't get this to work?

@ellisowen

This comment has been minimized.

Show comment
Hide comment
@ellisowen

ellisowen Aug 19, 2014

Contributor

The sphinx docs look quite different to mine... a couple of questions in line below:
(The other stuff I will address and push up a new version this evening).


OK ... finally I was able to build the Sphinx docs for this ... I've put them here for reference:
http://www.mpi-hd.mpg.de/personalhomes/deil/public/gammapy/issue_151/tutorials/npred/predicted_counts.html

  • I thought now the predicted counts agreed completely ... now I see again gtmodel_npred = 265 and gammapy_npred = 281. Are these simply outdated numbers or are results non-consistent again?

These values have never agreed exactly - from what I understand, the method I implement differs from the method used by gtmodel (I don't understand the full details of how this works) - so we thought before values within 10% would be fine?

  • The gtmodel image looks smoother ... does the PSF convolution work properly?

My PSF parameters may be different - I looked for a few hours yesterday to try and find out what kernel size, correction radius etc I should use, but didn't find this info yet... do you know what would be best?

  • The significance images don't show Vela? In any case I'd suggest removing them from this tutorial or putting it at the end, because the npred computation is a separate first step and the computation of a significance image is a separate later step.

This was due to the URL being wrong in gammapy.datasets.load (I couldn't check this before while the gammapy-extra PR wasn't merged). This is now fixed and shows correct images.

  • I know we've discussed this before and there was some problem, but for sky images in the docs we really should use aplpy.FITSFigure.
    It should just work if you pass it a filename or an HDU.
    Could you please give this another try and paste the exact code / error message here if you can't get this to work?

The issue here is that aplpy doesn't work on my laptop (normally it just freezes and then throws some time out error or memory error after 10 mins or so). This is why we decided to make a second PR in the days when I'm back in HD & can work on my desktop, and stick with matplotlib for now as I need this code working & merged asap.

Contributor

ellisowen commented Aug 19, 2014

The sphinx docs look quite different to mine... a couple of questions in line below:
(The other stuff I will address and push up a new version this evening).


OK ... finally I was able to build the Sphinx docs for this ... I've put them here for reference:
http://www.mpi-hd.mpg.de/personalhomes/deil/public/gammapy/issue_151/tutorials/npred/predicted_counts.html

  • I thought now the predicted counts agreed completely ... now I see again gtmodel_npred = 265 and gammapy_npred = 281. Are these simply outdated numbers or are results non-consistent again?

These values have never agreed exactly - from what I understand, the method I implement differs from the method used by gtmodel (I don't understand the full details of how this works) - so we thought before values within 10% would be fine?

  • The gtmodel image looks smoother ... does the PSF convolution work properly?

My PSF parameters may be different - I looked for a few hours yesterday to try and find out what kernel size, correction radius etc I should use, but didn't find this info yet... do you know what would be best?

  • The significance images don't show Vela? In any case I'd suggest removing them from this tutorial or putting it at the end, because the npred computation is a separate first step and the computation of a significance image is a separate later step.

This was due to the URL being wrong in gammapy.datasets.load (I couldn't check this before while the gammapy-extra PR wasn't merged). This is now fixed and shows correct images.

  • I know we've discussed this before and there was some problem, but for sky images in the docs we really should use aplpy.FITSFigure.
    It should just work if you pass it a filename or an HDU.
    Could you please give this another try and paste the exact code / error message here if you can't get this to work?

The issue here is that aplpy doesn't work on my laptop (normally it just freezes and then throws some time out error or memory error after 10 mins or so). This is why we decided to make a second PR in the days when I'm back in HD & can work on my desktop, and stick with matplotlib for now as I need this code working & merged asap.

@ellisowen

This comment has been minimized.

Show comment
Hide comment
@ellisowen

ellisowen Aug 19, 2014

Contributor

OK, so I made the changes you suggested apart from a couple of changes to the tutorial page (mentioned above) which I would prefer to address in a future PR when this is less time-critical, if that's OK?

I also left a couple of questions above.

The Travis CI failure appears to relate to a time-out error in build_sphinx as far as I can tell.

Contributor

ellisowen commented Aug 19, 2014

OK, so I made the changes you suggested apart from a couple of changes to the tutorial page (mentioned above) which I would prefer to address in a future PR when this is less time-critical, if that's OK?

I also left a couple of questions above.

The Travis CI failure appears to relate to a time-out error in build_sphinx as far as I can tell.

@cdeil

This comment has been minimized.

Show comment
Hide comment
@cdeil

cdeil Aug 20, 2014

Member

The reason I got weird results before was that I had old example data files in my cache ... after rm -r ~/.astropy I get sensible images.

There's a few things I'd like to improve about this tutorial (e.g. use aplpy for plotting) and the code (I plan to add a Dataset class that combine counts, exposure and psf). And we should try to get really consistent npred results (within 1%).
But it's better to do this later in separate PRs ... this is ready to be merged.

Thanks for your hard work and patience with the review of this PR!

Member

cdeil commented Aug 20, 2014

The reason I got weird results before was that I had old example data files in my cache ... after rm -r ~/.astropy I get sensible images.

There's a few things I'd like to improve about this tutorial (e.g. use aplpy for plotting) and the code (I plan to add a Dataset class that combine counts, exposure and psf). And we should try to get really consistent npred results (within 1%).
But it's better to do this later in separate PRs ... this is ready to be merged.

Thanks for your hard work and patience with the review of this PR!

cdeil added a commit that referenced this pull request Aug 20, 2014

@cdeil cdeil merged commit 9e4164e into gammapy:master Aug 20, 2014

1 check failed

continuous-integration/travis-ci The Travis CI build failed
Details
@ellisowen

This comment has been minimized.

Show comment
Hide comment
@ellisowen

ellisowen Aug 20, 2014

Contributor

Great! - I agree that future PRs will be better to make these improvements, which I will work on when I am back.

Contributor

ellisowen commented Aug 20, 2014

Great! - I agree that future PRs will be better to make these improvements, which I will work on when I am back.

@cdeil cdeil changed the title from Npred cube2 to Improve npred cube functionality Apr 8, 2015

@cdeil cdeil added the feature label Apr 8, 2015

@cdeil cdeil added this to the 0.1 milestone Apr 8, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment