Integration with Images.jl?

[timholy]

I notice there's no integration with Images.jl here, and I'm curious about the reasons. If Images.jl doesn't actually offer anything of value to you, what would it take to make it useful? (I'm not saying I expect that it already has every algorithm you'd ever need, but I'd be equally surprised if there's nothing of value---if so, that's clearly a deficiency in Images.jl.) If Images.jl's I/O architecture isn't compatible with your needs, again it would be interesting to know what would need to be done to make it work. Images.jl is currently being used by the computer-vision and biomedical imaging folks, but I'm not aware of any astronomy people, and it would be great to make sure your needs are also met.

For I/O integration, see docs here:
https://github.com/timholy/Images.jl/blob/master/doc/extendingIO.md
The hope would be that a couple of stub functions in a file starting with using FITSIO would allow easy integration. If you need more sophisticated control over I/O than Images currently offers, well, that would be something we could talk about.

[kbarbary]

I think the main reason is that there are not yet any Julia astro packages for actually doing anything with images. So, no one has really worried about a type for conveniently representing astronomical images in memory (beyond simple arrays), or about the image operations that Images.jl provides.

I did a little exploratory work on a package to actually do some analysis on images and I mean to get back to it at some point. I think any such package will want to be compatible with Image.jl's AbstractImage type so that it can take advantage of any astro-relevant functionality in Images.jl. (At a quick glance, the "filtering" functions are something that would definitely be relevant.) When I get back to that, I'll be sure to look closely at the Image types and ping you if there's anything missing (from an optical/near-IR astro perspective anyway).

Thanks for the pointer to the Images.jl I/O infrastructure; I wasn't aware of it. Is the idea that this could be a sub-module of Images so that FITSIO would only be used if installed? E.g.

using Images
using Images.FITS

img = imread("filename.fits")

[timholy]

Even easier (from a user perspective):

using Images
img = imread("filename.fits")

The extension alone (or even better, a magic number in the file if FITS files have one) would be enough to trigger loading of FITSIO via a short wrapper.

[timholy]

One thing I should point out is that I'm wondering if you folks might need enhancements to imread. I imagine you folks must have some pretty massive images, and might want to read portions of them at any one time. I already use mmap to "load" huge images (>1TB) in my own work, but if for some reason you can't use mmap then I wonder if we'll need a mechanism for passing a region-of-interest to imread.

[kbarbary]

Ah, yes. There's a couple things about FITS to note:

1. FITS binary data is big-endian by definition, so it doesn't work great with mmap. (The cfitsio library does the byte-swapping when you read the data.) Being able to pass Ranges to imread would be good.
2. FITS files have multiple "header data units" (HDUs), and the image of interest could be in any HDU. Being able to specify either the name or number of the HDU would be necessary.

[timholy]

I'm not surprised to hear it could be complicated.

It's possible that integrating with imread will be more pain than it's worth, and that your best bet is to use a separate API (as you do now). You could return the data as an Image, of course. Most if not all of the algorithms in Images.jl do work with plain arrays, so there's no requirement for a wrapper-type, but you can set up more things that "just work" if you're allowed to attach metadata to the image (which is basically what the Image type does for you).

Another thing that I imagine astronomers might want would be some extensions to Color.jl, for e.g., hyperspectral imaging. Images already defines some extensions to Color, and I would have no problem with the idea of adding more for hyperspectral applications. (It just hasn't come up before.)

[giordano]

I've just created the package AstroImages.jl, which should provide integration of astronomical images with popular images and plotting packages. This is very basic, at the moment it provides only a FileIO.load method to read an extension from a FITS file and a new type, AstroImage, to convert the extension to Matrix{Gray}.

I think the discussion should continue to JuliaAstro/AstroImages.jl#1