Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High-level function: make_sky_image #30

Merged
merged 52 commits into from
Jun 30, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
8add025
Add function make_sky_image
adl1995 Jun 22, 2017
eefb5d2
Introduce hips.utils.testing
cdeil Jun 24, 2017
32ce875
Add WCSGeometry.fits_header property
cdeil Jun 24, 2017
06e676e
Continue make_sky_image and draw_sky_image implementation
cdeil Jun 24, 2017
a5ac131
Get hips-extra for travis-ci
cdeil Jun 24, 2017
1dcc811
Merge branch 'master' of https://github.com/hipspy/hips into tiles.draw
adl1995 Jun 26, 2017
120ff07
Add skeleton of drawing code
adl1995 Jun 26, 2017
818a910
Update drawing function
adl1995 Jun 27, 2017
91e7e39
Store FITS data as float type
adl1995 Jun 27, 2017
c838f64
Update drawing code
adl1995 Jun 27, 2017
3acf68e
Update corner computation
adl1995 Jun 28, 2017
243ef55
Add dst property in HipsTileMeta class
adl1995 Jun 28, 2017
53a0df5
Reformat some code
adl1995 Jun 28, 2017
7897504
Fix drawing code
adl1995 Jun 29, 2017
1e4bb3b
Add property skycoord_corner to HipsTileMeta
adl1995 Jun 29, 2017
c3d3d27
Fix link to WCSGeometry class in docs
adl1995 Jun 29, 2017
e487e44
Update drawing code
adl1995 Jun 29, 2017
42ceb7a
Get HiPS order from HipsSurveyProperties class
adl1995 Jun 29, 2017
a32ba1b
Add example in Getting Started page
adl1995 Jun 29, 2017
f7625a9
Add property access_url in HipsSurveyProperties
adl1995 Jun 29, 2017
05b8ac8
Add test case for access_url property
adl1995 Jun 29, 2017
1f67310
Add function apply_projection in HipsTileMeta
adl1995 Jun 29, 2017
8adff08
Update drawing code
adl1995 Jun 29, 2017
9c82bc4
Use assert_allclose instead of assert
adl1995 Jun 30, 2017
9c5e687
Add missing import
adl1995 Jun 30, 2017
bee4bcd
Document frame parameter in HipsTileMeta
adl1995 Jun 30, 2017
c7d5db5
Remove function apply_projection from HipsTileMeta
adl1995 Jun 30, 2017
69ae764
Add function apply_projection in SimpleTilePainter
adl1995 Jun 30, 2017
9f2e5fa
Add test case for HipsTileMeta
adl1995 Jun 30, 2017
9e073c9
Remove docstring
adl1995 Jun 30, 2017
4571dd8
Add comment on data type conversion
adl1995 Jun 30, 2017
a0759cb
Remove test case for SimpleTilePainter class
adl1995 Jun 30, 2017
309a75a
Pass fits_header to fits.PrimaryHDU function
adl1995 Jun 30, 2017
2846fca
Merge HipsTile and HipsTileMeta within same file
adl1995 Jun 30, 2017
a7d4ba4
Update test case for HipsTileMeta class
adl1995 Jun 30, 2017
3c8204a
Add property projection in SimpleTilePainter
adl1995 Jun 30, 2017
9b79b9d
Update test_skycoord_corners in TestHipsTileMeta
adl1995 Jun 30, 2017
a0ad4cb
Rename all_sky to sky_sky
adl1995 Jun 30, 2017
a9b0984
Rename draw_tile to painter
adl1995 Jun 30, 2017
de55663
Add function frames in utils/healpix.py
adl1995 Jun 30, 2017
186286c
Update test_skycoord_corners in TestHipsTileMeta
adl1995 Jun 30, 2017
451b32a
Update read classmethod in HipsTile class
adl1995 Jun 30, 2017
c34d406
Update requires_hips_extra decorator function
adl1995 Jun 30, 2017
112fb37
Update example for HipsTileMeta
adl1995 Jun 30, 2017
69e09af
Skip example from doctest
adl1995 Jun 30, 2017
a4b6846
Remove attribute self.pt from SimpleTilePainter
adl1995 Jun 30, 2017
5c85e12
Add property tile_access_url in HipsSurveyProperties
adl1995 Jun 30, 2017
145bcaf
Add attribute frames in HipsSurveyProperties
adl1995 Jun 30, 2017
5fe3ed0
Some code reformatting
adl1995 Jun 30, 2017
880d4a2
Add test case test_wcs_healpix_pixel_indices
adl1995 Jun 30, 2017
e02407e
Fixes and cleanup of make_sky_image
cdeil Jun 30, 2017
b739539
Fix make_sky_image implementation
cdeil Jun 30, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ env:
- SETUP_XVFB=True
- DEBUG=True
- ASTROPY_USE_SYSTEM_PYTEST=1
- FETCH_HIPS_EXTRA=true

matrix:
# Make sure that egg_info works without dependencies
Expand Down Expand Up @@ -80,6 +81,12 @@ install:
- git clone git://github.com/astropy/ci-helpers.git
- source ci-helpers/travis/setup_conda.sh

- if $FETCH_HIPS_EXTRA; then
git clone https://github.com/hipspy/hips-extra.git $HOME/hips-extra;
export HIPS_EXTRA=${HOME}/hips-extra;
fi


script:
- $MAIN_CMD $SETUP_CMD

Expand Down
28 changes: 27 additions & 1 deletion docs/getting_started.rst
Original file line number Diff line number Diff line change
@@ -1,9 +1,35 @@
.. include:: references.txt

.. doctest-skip-all

.. _gs:

***************
Getting started
***************

Coming soon ...
The example below shows a high level use case of the ``hips`` package.
It fetches a HiPS tile from a remote URL and draws it on a sky image.
Then it saves it on local disk in FITS file format.

::

from astropy.io import fits
from astropy.coordinates import SkyCoord
from astropy.tests.helper import remote_data
from hips.utils import WCSGeometry
from hips.draw import make_sky_image
from hips.tiles import HipsSurveyProperties

geometry = WCSGeometry.create(
skydir=SkyCoord(0, 0, unit='deg', frame='galactic'),
shape=(1000, 2000), coordsys='GAL',
projection='AIT', cdelt=0.01, crpix=(1000, 500),
)
url = 'https://raw.githubusercontent.com/hipspy/hips-extra/master/datasets/samples/DSS2Red/properties'
hips_survey = HipsSurveyProperties.fetch(url)

data = make_sky_image(geometry, hips_survey)

hdu = fits.PrimaryHDU(data=data, header=geometry.fits_header)
hdu.writeto('my_image.fits')
126 changes: 126 additions & 0 deletions hips/draw/simple.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,131 @@
# Licensed under a 3-clause BSD style license - see LICENSE.rst
"""HiPS tile drawing -- simple method."""
from typing import Generator, Any
import numpy as np
from skimage import transform as tf
from ..tiles import HipsSurveyProperties, HipsTile, HipsTileMeta
from ..utils import WCSGeometry, compute_healpix_pixel_indices

__all__ = [
'draw_sky_image',
'make_sky_image',
'SimpleTilePainter'
]


def draw_sky_image(geometry: WCSGeometry, tiles: Generator[HipsTile, Any, Any]) -> np.ndarray:
"""Draw sky image using the simple and quick method.

Parameters
----------
geometry : `~hips.utils.WCSGeometry`
An object of WCSGeometry
tiles : List[HipsTile]
A list of HipsTile

Returns
-------
np.ndarray
Returns a numpy array containing all HiPS tiles projected onto it
"""
# TODO: Fix type annotation issue
sky_sky = np.zeros(geometry.shape)
for tile in tiles:
painter = SimpleTilePainter(geometry, tile)
sky_sky += painter.warp_image()
return sky_sky


class SimpleTilePainter:
"""Paint a single tile using a simple projective transformation method.

The algorithm implemented is described here: :ref:`drawing_algo`.

Parameters
----------
geometry : `~hips.utils.WCSGeometry`
An object of WCSGeometry
tile : `HipsTile`
An object of HipsTile
"""

def __init__(self, geometry: WCSGeometry, tile: HipsTile) -> None:
self.geometry = geometry
self.tile = tile

@property
def projection(self) -> tf.ProjectiveTransform:
"""Estimate projective transformation on a HiPS tile"""
corners = self.tile.meta.skycoord_corners.to_pixel(self.geometry.wcs)
src = np.array(corners).T.reshape((4, 2))
dst = self.tile.meta.dst
pt = tf.ProjectiveTransform()
pt.estimate(src, dst)
return pt

def warp_image(self) -> np.ndarray:
"""Warp a HiPS tile and a sky image"""
return tf.warp(
self.tile.data,
self.projection,
output_shape=self.geometry.shape,
preserve_range=True,
)


def fetch_tiles(healpix_pixel_indices: np.ndarray, order: int, hips_survey: HipsSurveyProperties) -> 'HipsTile':
"""Fetch HiPS tiles from a remote URL.

Parameters
----------
healpix_pixel_indices : np.ndarray
A list of HEALPix pixel indices
order : int
Order of the HEALPix map
hips_survey : HipsSurveyProperties
An object of HipsSurveyProperties

Returns
-------
'HipsTile'
Returns an object of HipsTile
"""
for healpix_pixel_index in healpix_pixel_indices:
tile_meta = HipsTileMeta(
order=order,
ipix=healpix_pixel_index,
frame=hips_survey.astropy_frame,
file_format='fits',
)
tile = HipsTile.fetch(tile_meta, hips_survey.tile_access_url + tile_meta.filename)
yield tile


def make_sky_image(geometry: WCSGeometry, hips_survey: HipsSurveyProperties) -> np.ndarray:
"""Make sky image: fetch tiles and draw.

The example for this can be found on the :ref:`gs` page.

Parameters
----------
geometry : `~hips.utils.WCSGeometry`
Geometry of the output image
hips_survey : `~hips.HipsSurveyProperties`
HiPS survey properties

Returns
-------
data : `~numpy.ndarray`
Output image pixels
"""
healpix_pixel_indices = compute_healpix_pixel_indices(
wcs_geometry=geometry,
order=hips_survey.hips_order,
healpix_frame=hips_survey.astropy_frame,
)
# TODO: this isn't a good API. Will become better when we have a cache.
tiles = fetch_tiles(healpix_pixel_indices, hips_survey.hips_order, hips_survey)

image_data = draw_sky_image(geometry, tiles)

return image_data
54 changes: 54 additions & 0 deletions hips/draw/tests/test_simple.py
Original file line number Diff line number Diff line change
@@ -1 +1,55 @@
# Licensed under a 3-clause BSD style license - see LICENSE.rst
import numpy as np
from numpy.testing import assert_allclose
from astropy.utils.data import get_pkg_data_filename
from astropy.tests.helper import remote_data
from ..simple import make_sky_image, draw_sky_image
from ...tiles import HipsSurveyProperties, HipsTileMeta, HipsTile
from ...utils.testing import get_hips_extra_file, make_test_wcs_geometry, requires_hips_extra


def get_test_tiles():
filename = get_pkg_data_filename('../../tiles/tests/data/properties.txt')
hips_survey = HipsSurveyProperties.read(filename)

tile1 = HipsTile.read(
meta=HipsTileMeta(order=3, ipix=450, file_format='fits', frame=hips_survey.astropy_frame,
tile_width=512),
filename=get_hips_extra_file('datasets/samples/DSS2Red/Norder3/Dir0/Npix450.fits'),
)

tile2 = HipsTile.read(
meta=HipsTileMeta(order=3, ipix=451, file_format='fits', frame=hips_survey.astropy_frame,
tile_width=512),
filename=get_hips_extra_file('datasets/samples/DSS2Red/Norder3/Dir0/Npix451.fits'),
)

return [tile1, tile2]


@requires_hips_extra()
def test_draw_sky_image():
geometry = make_test_wcs_geometry(case=2)
tiles = get_test_tiles()
data = draw_sky_image(geometry, tiles)

assert data.shape == geometry.shape
assert data.dtype == np.float64
assert_allclose(np.sum(data), 4575235421.5126467)
assert_allclose(data[400, 500], 2866.0101409848185)
assert_allclose(data[400, 501], 2563.6916727348043)


@remote_data
def test_make_sky_image():
url = 'https://raw.githubusercontent.com/hipspy/hips-extra/master/datasets/samples/DSS2Red/properties'
hips_survey = HipsSurveyProperties.fetch(url)
geometry = make_test_wcs_geometry(case=2)
data = make_sky_image(geometry, hips_survey)
assert data.shape == geometry.shape
assert data.dtype == np.float64
assert_allclose(data[200, 994], 3717.10091363)
assert_allclose(data[200, 995], 3402.55292158)


# TODO: add tests for SimpleTilePainter with asserts on the intermediate computed things.
1 change: 0 additions & 1 deletion hips/tiles/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# Licensed under a 3-clause BSD style license - see LICENSE.rst
"""Classes and functions to manage HiPS tiles."""
from .tile import *
from .tile_meta import *
from .description import *
from .surveys import *
21 changes: 21 additions & 0 deletions hips/tiles/description.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ class HipsSurveyProperties:
>>> hips_survey_property.base_url
'http://alasky.u-strasbg.fr/DSS/DSSColor'
"""
hips_to_astropy_frame_mapping = OrderedDict([
('equatorial', 'icrs'),
('galactic', 'galactic'),
('ecliptic', 'ecliptic'),
])
"""HIPS to Astropy SkyCoord frame string mapping."""

def __init__(self, data: OrderedDict) -> None:
self.data = data
Expand Down Expand Up @@ -99,6 +105,11 @@ def hips_frame(self) -> str:
"""HiPS coordinate frame (`str`)."""
return self.data['hips_frame']

@property
def astropy_frame(self) -> str:
"""Astropy coordinate frame (`str`)."""
return self.hips_to_astropy_frame_mapping[self.hips_frame]

@property
def hips_order(self) -> int:
"""HiPS order (`int`)."""
Expand All @@ -108,3 +119,13 @@ def hips_order(self) -> int:
def tile_format(self) -> str:
"""HiPS tile format (`str`)."""
return self.data['hips_tile_format']

@property
def access_url(self):
"""HiPS access url"""
return self.data['moc_access_url'].rsplit('/', 1)[0]

@property
def tile_access_url(self):
"""Tile access URL for a HiPS surveys"""
return self.access_url + '/Norder' + str(self.hips_order) + '/Dir0/'
9 changes: 9 additions & 0 deletions hips/tiles/tests/test_description.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,17 @@ def test_hips_version(self):
def test_hips_frame(self):
assert self.hips_survey_property.hips_frame == 'equatorial'

def test_astropy_frame(self):
assert self.hips_survey_property.astropy_frame == 'icrs'

def test_hips_order(self):
assert self.hips_survey_property.hips_order == 9

def test_tile_format(self):
assert self.hips_survey_property.tile_format == 'jpeg'

def test_access_url(self):
assert self.hips_survey_property.access_url == 'http://alasky.u-strasbg.fr/DSS/DSSColor'

def test_tile_access_url(self):
assert self.hips_survey_property.tile_access_url == 'http://alasky.u-strasbg.fr/DSS/DSSColor/Norder9/Dir0/'
40 changes: 37 additions & 3 deletions hips/tiles/tests/test_tile.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Licensed under a 3-clause BSD style license - see LICENSE.rst
from numpy.testing import assert_equal
import numpy as np
from astropy.tests.helper import remote_data
from ..tile import HipsTile
from ..tile_meta import HipsTileMeta
from numpy.testing import assert_allclose
from numpy.testing import assert_equal

from ..tile import HipsTile, HipsTileMeta


class TestHipsTile:
Expand Down Expand Up @@ -41,3 +43,35 @@ def test_fetch_read_write_fits(self, tmpdir):
# print(tile2.data.sum())
# print((tile == tile2).all())
# assert tile == tile2


class TestHipsTileMeta:
@classmethod
def setup_class(cls):
cls.meta = HipsTileMeta(order=3, ipix=450, file_format='fits', frame='icrs', tile_width=512)

def test_path(self):
assert str(self.meta.path) == 'hips/tiles/tests/data'

def test_filename(self):
assert self.meta.filename == 'Npix450.fits'

def test_full_path(self):
assert str(self.meta.full_path) == 'hips/tiles/tests/data/Npix450.fits'

def test_nside(self):
assert self.meta.nside == 8

def test_dst(self):
dst = np.array([[511, 0], [511, 511], [0, 511], [0, 0]])
assert_allclose(self.meta.dst, dst)

def test_skycoord_corners(self):
assert_allclose(self.meta.skycoord_corners.data.lat.deg, [-24.624318, -30., -35.685335, -30.])
assert_allclose(self.meta.skycoord_corners.data.lon.deg, [264.375, 258.75, 264.375, 270.])
assert self.meta.skycoord_corners.frame.name == 'icrs'

meta = HipsTileMeta(order=3, ipix=450, file_format='fits', frame='galactic', tile_width=512)
assert_allclose(meta.skycoord_corners.data.lat.deg, [-24.624318, -30., -35.685335, -30.])
assert_allclose(meta.skycoord_corners.data.lon.deg, [264.375, 258.75, 264.375, 270.])
assert meta.skycoord_corners.frame.name == 'galactic'
Loading