New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement I/O methods for gammapy.maps #1093

Merged
merged 45 commits into from Aug 11, 2017

Conversation

Projects
None yet
2 participants
@woodmd
Member

woodmd commented Jul 28, 2017

This PR fleshes out more of the FITS I/O related functionality in gammapy.maps and adds some new features to HpxMapND:

  • Implement read and write methods for WcsMapND and HpxMapND.
  • Maps can be written to a sparse format by calling write with sparse=True. This writes a BINTABLE with one row per pixel. The format for HPX maps is documented on the gamma-astro-data-formats page. WCS maps use a similar convention which I will try to document in the next few weeks. WcsMapND.read and HpxMapND.read can instantiate a map from a FITS HDU in either the sparse or non-sparse format.
  • Support for instantiating HpxMapND with a multi-resolution geometry (i.e. different pixel size in each image plane). This is implemented by allocating an array with an image dimension with size equal to the largest image plane in the map and filling array elements outside the geometry in a given image plane with guard values. It's not particularly memory efficient but it should be useful for testing multi-resolution maps until the sparse map classes are fully implemented.

@cdeil cdeil added the feature label Jul 28, 2017

@cdeil cdeil added this to the 0.7 milestone Jul 28, 2017

Show outdated Hide outdated gammapy/maps/base.py Outdated
@cdeil

cdeil approved these changes Jul 28, 2017

I left one inline comment that the test fail on Windows / Appveyor should be fixed or marked xfail.
Otherwise, I'd suggest you merge this whenever you want, and then I and others will try out gammapy.maps in the coming months and give feedback.

@woodmd

This comment has been minimized.

Show comment
Hide comment
@woodmd

woodmd Jul 29, 2017

Member

Thanks for taking a look. I saw that error and was also stumped by it. Unfortunately it's not easy for me to debug not having access to a windows machine. Is there a way to mark a test with xfail for a specific platform (i.e. windows)?

Member

woodmd commented Jul 29, 2017

Thanks for taking a look. I saw that error and was also stumped by it. Unfortunately it's not easy for me to debug not having access to a windows machine. Is there a way to mark a test with xfail for a specific platform (i.e. windows)?

@woodmd

This comment has been minimized.

Show comment
Hide comment
@woodmd

woodmd Aug 11, 2017

Member

I went ahead and took the opportunity to include some more updates with this PR. Some of the additional changes:

  • Added framework to support writing files in formats that are backwards compatible with existing software like the Fermi ScienceTools. When you write a file you can set the conv parameter to enforce that the file is written in the appropriate format. This mostly comes down to restricting the name of the BANDS table. Ideally in the future all relevant software will be updated to support the formats documented in GADF but in the meantime we will probably need to keep some mechanism like this. Note that we have started to implement partial support for some of the GADF conventions in the Fermi Science Tools.
  • Implemented reproject method for WCS-based maps. There are a few cases that are still missing (e.g. 4D maps where the non-spatial axes are not aligned). One issue that arose was the lack of a method for wrapping a reprojection at the edge of an all-sky CAR map (e.g. like the fermi IEM). I opened astrofrog/reproject#135 to request this feature. For the moment I wrote some functions in gammapy.maps.reproject which can handle this but eventually it would be nice if this was included in the reproject package.
  • Added methods for iterating over the contents of a map.
  • Various improvements to docstrings and sphinx documentation pages.

The original test failure on Windows / Appveyor also seems to be resolved now (although I'm not sure what I did to fix it).

Member

woodmd commented Aug 11, 2017

I went ahead and took the opportunity to include some more updates with this PR. Some of the additional changes:

  • Added framework to support writing files in formats that are backwards compatible with existing software like the Fermi ScienceTools. When you write a file you can set the conv parameter to enforce that the file is written in the appropriate format. This mostly comes down to restricting the name of the BANDS table. Ideally in the future all relevant software will be updated to support the formats documented in GADF but in the meantime we will probably need to keep some mechanism like this. Note that we have started to implement partial support for some of the GADF conventions in the Fermi Science Tools.
  • Implemented reproject method for WCS-based maps. There are a few cases that are still missing (e.g. 4D maps where the non-spatial axes are not aligned). One issue that arose was the lack of a method for wrapping a reprojection at the edge of an all-sky CAR map (e.g. like the fermi IEM). I opened astrofrog/reproject#135 to request this feature. For the moment I wrote some functions in gammapy.maps.reproject which can handle this but eventually it would be nice if this was included in the reproject package.
  • Added methods for iterating over the contents of a map.
  • Various improvements to docstrings and sphinx documentation pages.

The original test failure on Windows / Appveyor also seems to be resolved now (although I'm not sure what I did to fix it).

@woodmd woodmd merged commit a77ba74 into gammapy:master Aug 11, 2017

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment