-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ideas/Wishlist for AstroImages #29
Comments
To add, there's a struct in CCDReduction.jl ( |
I wrote a using Plots
img = # ...
heatmap(img, clims=zscale) or just clo, chi = zscale(img) |
Thanks @mileslucas! I'll take a look there as well. How does broadcasting work for headers? |
Awkwardly. https://github.com/JuliaAstro/CCDReduction.jl/blob/dc83e241d54604987377360a799b17a3585e68d2/src/ccddata.jl#L90-L102 is the implementation, based off the docs here: https://docs.julialang.org/en/v1/manual/interfaces/#Selecting-an-appropriate-output-array |
I'm all for the external viewer incorporation. I've really grown to like using DS9 for diagnostic plotting. I have some open ideas I've yet to pursue here- JuliaAstro/SAOImageDS9.jl#4, JuliaAstro/SAOImageDS9.jl#3 |
Nice, I hadn't seen the DS9 package! I'd like to understand more what you mean by broadcasting headers. Do you mean broadcasting values across multiple headers? Or doing something special with headers when broadcasting images? |
Sorry for the confusion- what I'm doing with CCDData here is copying the headers when broadcasting the image data. So for example if you had a FITS image loaded and you did |
Ah okay thanks! I understand now. |
Here are the main image types defined by the libraries listed above: EasyFITS.jl: struct FitsImage{T,N} <: DenseArray{T,N}
arr::Array{T,N}
hdr::FitsHeader
end AstroImages.jl: struct AstroImage{T<:Real,C<:Color, N, P}
data::NTuple{N, Matrix{T}}
minmax::NTuple{N, Tuple{T,T}}
wcs::NTuple{N, WCSTransform}
property::Properties{P}
end CCDReduction.jl: abstract type AbstractCCDData{T} <: AbstractMatrix{T} end
struct CCDData{T,M<:AbstractMatrix{T}} <: AbstractCCDData{T}
data::M
hdr::FITSHeader
end DirectImages.jl: struct DirectImage{T,N,A<:AbstractArray} <: AbstractArray{T,N}
data::A
headers::OrderedDict{Symbol,CommentedValue}
end Observations:
Opinions:
|
So does CCDData :) |
Why |
Why not leave it parametric? There should be equivalent dict-based interfaces for fits headers and ordered dicts, and if not we need to enable that in FITSIO.jl |
Thanks, I missed that!
I guess you are right, there's no good reason to prevent people using e.g. sparse arrays. |
The three implementations I can think of are: vectors (like FTSHeader), Dicts, and named tuples. Using named tuples might improve type stability in some cases since the type of every header value would be known, but at the cost of a lot of compiler latency. |
I realized that |
One question I have for the maintainers of FITSIO is about mutable struct FITSHeader
keys::Vector{String}
values::Vector{Any}
comments::Vector{String}
map::Dict{String, Int}
# ...
end Is values::Vector{Union{Bool,Int,Float32,Nothing,InlineString127} where InlineString127 comes from InlineStrings.jl. If we could change that, then code accessing values from FITSHeaders wouldn't be type unstable. In my testing Julia is able to do union splitting with that element type and doesn't box things. Edit: I believe inline strings could be used because fits string headers are limited to 68 characters (ASCII, I assume?) |
I did some testing on restricting the range of header types. HType = Union{Bool,Int,Float64,Nothing,String}
function test1(v,i)
v[i]*"abc"
end
dat = HType[1,nothing,false,0.1,"abcd"]
@code_typed test1(dat, 5) Gives:
So the compiler is able to generate fast code consisting of only a On the other hand, if we reuse FITSIO.FITSHeader and the type of
which is fully generic and always allocates. IMO it would be reasonable to restrict the headers to only valid FITS header types to get this improvement. Edit: I made that modification to FITSIO and everything seems to work well. Floats and Integers get automatically converted to Float64 and Int and unsupported values display a warning and get stored as a string. |
Regarding the AstroImages.jl/src/AstroImages.jl Lines 169 to 176 in 1b43abc
And related to this, this render function would always call extrema on the whole image array, which would take a lot every time you want to convert to a Matrix{Color} , thus AstroImage has the minmax field purely for performance reasons, not else: #13. But I'm happy for these details to be revised.
|
Thanks @giordano! The lazy conversion / colorview makes a lot of sense. What happens if the user modifies the array? Maybe there is a way to catch this and recalculate min/max on the next render? |
Thinking aloud, if we use code like https://github.com/JuliaAstro/CCDReduction.jl/blob/dc83e241d54604987377360a799b17a3585e68d2/src/ccddata.jl#L12 from CCDReduction, we could add a flag like Then, I'm not sure what the performance impact of that would be though. I don't think the flag would have to atomic. If another thread modified the data between calculating |
Support for Multiple HDUs
I personally use muti-extension fits files but each HDU is conceptually a different array. For other people's use cases, are there times grouping together multiple HDUs in a single struct would be better than a vector/tuple of images? |
That's a good question and I hadn't anticipated such need 😕 This is implicitly assuming the data array won't be modified in-place.
I agree with that, each extension is in principle independent. However, the reason why |
Ah of course, RGB 😃 Maybe this could be split into a separate wrapper type that groups together N AstroImages and calling For full generality, we could support other color spaces e.g. |
Of course keeping multiple channels in separate arrays/dimensions gets away from the design choices of the Images ecosystem where they are packed into the array elements. I'm not sure what tradeoff is best. |
Yup, I'd be OK with having another struct for handling colours.
Sure. RGB is the only colour mapping this package is currently specialised for, but it isn't limited to that one. Other options would of course be great! |
For accessing header comments, maybe a clearer syntax than |
If we use For example, And perhaps |
For displaying images, I propose we make an explicit function called something like Wrapping in an AstroImage would opt into automatic display at the REPL (and in Jupyter, etc) which could just fall through to To configure the default display, we could then have a global configuration (a bit like a plot theme but far simpler). |
My current plan is to continue discussions in this thread and build consensus. If/once we're all happy with the basics, I'll open a pull request back here for more detailed review. |
Here is a demonstration with more curved coordinates: julia> plot(WCSGrid(img), aspectratio=1, framestyle=:box, color=:black) To make this work, I've created a struct called Once you have created a grid, you can for example get the ticks for where it intersects the axes via the helper julia> g = WCSGrid(img)
WCSGrid(WCSTransform(naxis=2), (1, 1600, 1, 1600), (1, 2), (1, 1))
julia> wcsticks(g, 1)
([828.5121645296939], ["-2ʰ"])
julia> wcsticks(g, 2)
([1542.4204273078626, 1012.6018518589875, 496.24868471405944], ["183°", "200°", "216°"]) Or vectors of x and y points defining the gridlines using: julia> xs, ys = wcsgridlines(g) I'm still not sure how to handle laying out the grids for images where the coordinates wrap around part way through. |
This looks really great! One question I have the relates to my work is can we make this deal with polarization (Stokes I, Q, U, V and maybe circular polarization as well) in a nice way? This will be important for radio astronomy where the polarization contains a ton of information about the emitting electron populations and can e.g. be used to differentiate between different types of accretion. You probably also want to change plotting for polarized images. Usually, linear polarization is denoted with ticks overlayed on the stokes I emission. |
Thanks @ptiede! I hope we can. Is that information stored in the WCS headers? |
I don't think this will fit in the header, but I am not too familiar with them. I usually think of polarization as being something akin to color, so each image pixel is really four numbers. How I usually dealt with this in the past was to use a StructArray with a custom Stokes vector type that contains I,Q,U,V. So I think to make polarization work you just need to ensure that the images can be stored in some dimension. I'll try to grab a polarized CLEAN image and send it your way! |
Here are some updates showing different coordinate systems. Cartesian projected galactic coodinates with rectilinear gridplot(mws, grid=true, gridcolor=:white, background=:black, framestyle=:box) Mollweide projected galactic coodinatesplot(ovro, grid=true, gridcolor=:cyan, framestyle=:box, background="#222") In this case, the x-ticks have gotten all bunched up at the bottom. There's really not much we can do about this other than adding a heuristic to remove plot ticks very close together. For this reason, I'm keeping the option to directly annotate the grid though it needs a bit more work: plot(ovro, grid=true, gridcolor=:cyan, framestyle=:box, background="#222", annotategrid=true, ticks=[]) |
As Miles has pointed out, some of this machinery already exists when using views. For instance: julia> A = randn(100,100,10)
julia> v = view(A, :, :, 5)
julia> parentindices(v)
(Base.Slice(Base.OneTo(100)), Base.Slice(Base.OneTo(100)), 5) So maybe there is a way to re-use this for general slicing (which copies) in addition to creating views. |
Another option is to do this ourselves and store the following fields in the AstroImage struct:
This is sort of creating our own OffsetArrays, and the OffsetArrays.jl code is quite daunting. |
What are your thoughts on editing the recipe to add xlim --> extrema(first(axes(image))
ylim --> extrema(second(axes(image)) (or, however you get the indices) This would eliminate the gridded whitespace in the Plots.jl images. Here's an example of what a plot looks like with these settings: https://juliaastro.github.io/PSFModels.jl/dev/examples/#Fitting-a-PSF |
Great point, yes I think we should have that in the recipe. This is doubly important when world coordinates are in play since they are calculated along the axes of the image. If the plot adds extra white space they can become incorrect. |
I've been working steadily on this and am making some good progress.
I also removed support for just I'm going to make a few demo notebooks to showcase everything, but here are a few previews. Loading a cube with axes galactic longitude, galactic latitude, velocity: fname = download("http://data.astropy.org/tutorials/FITS-cubes/reduced_TAN_C14.fits")
h = load(fname)
implot(h[Z=300], cmap=:turbo)
# Or just implot(h[:,:,300], cmap=:turbo) Plotting one spectrum extracted from the same cube (this recipe just comes from DimensionalData) plot(h[X=145,Y=130]) Should have lots to show by the next JuliaAstro call! |
Hi @ptiede , would love to test out one of your FITS files! |
I'm also interested in polarization plotting. My concern is I don't believe there is a standard for arraying polarized data. Perhaps there are standards in the radio community, but I don't know of any in the direct imaging community. I think this is better suited for some different recipes/functions s.t. we aren't tied down to a specific data layout. For example, given stokes Q and U images, we could plot the polarized intensity ( I'd be happy to give you some of my FITS images if you want to play around with them! |
I have no experience here, but I did see some allowances for polarization axes in FITS world coordinates (https://www.aanda.org/articles/aa/pdf/2002/45/aah3859.pdf , table 7 for example). Does that have any relevance?
Yes please! |
So for linear polarization, the EHT has a few ways to represent images. The one we presented in EHTC VII are of the form: The tick position angle is the EVPA which is 1/2atan(U/Q), the length of the tick is the absolute linear polarization in Jy (sqrt(Q^2+U^2)) and the color denotes the linear polarization fraction (sqrt(Q^2+U^2)/I). The greyscale image in the background is the total stokes I image. Now this is just for EHT and conventions are a bit of a mess but I think this is a reasonable starting point. For circular polarization, one thing I have seen is to replace the ticks with little ellipses and then change the ellipse's size depending on the circular polarization. For why circular polarization is interesting, well for the EHT it encodes interesting magnetic field properties around black holes. A recent and quite nice reference for this is https://arxiv.org/pdf/2104.11301.pdf |
Currently |
Yes, it seems like this is handled by the logic here. |
I did not know about this, thanks for pointing it out! That being said, I've never used WCS for any high-contrast work, and I haven't personally seen anyone who does. So I wouldn't rely on it.
Me too. |
Many high contrast imaging instruments I’ve used do have WCS headers attached to their raw data (GPI, NIRC2, HST have them, not sure about VLT-SPHERE or LBT LMIRCam) but I haven’t ever looked at polarization data from any of them in my work. I’ll track down some GPI cubes and see if they have the proper info in the headers. It’s a good point that we need to make sure all functionality doesn’t assume the data comes from FITS files with well formed headers. Right now the dimensions are labelled as X,Y,Z,Ax{4},Ax{5}, etc automatically. |
Thanks! |
After your comments @mileslucas and @ptiede, I think I have a solution for files without WCS info where we still want to give certain axes labels and/or special meaning, e.g. polarization, spectral, or time axes. I've made the img = AstroImage( randn(101, 101), (-50:50, -50:50)) # Or:
img = AstroImage( randn(101, 101), (X=-50:50, Y=-50:50)) constructs an AstroImage with dimensions centred around zero. Another example from a real image: We can use this for arbitrary cubes even if the WCS info isn't set: cube = AstroImage( randn(15,15,100,45), (X, Y, Ti, Spec))
# Extracting a spectrum:
p1 = plot(cube[X=1,Y=1,Ti=10,Spec=1:45])
# Plotting a wavelength slice:
p2 = implot(cube[Ti=10,Spec=35])
plot(p1, p2, layout=(2,1), size=(350,700)) Finally, we can use categorical axes to store polarization data in a nice way (edit: or rather, describe how the polarization data is stored in an existing cube) polcube = AstroImage( randn(15,15,3), (X=1:15, Y=1:15, Pol=[:I, :Q, :U]))
# Polarization plotting TBD To restore the previous behaviour of matching the dimensions to the WCS axes, we can pass So now the examples from earlier in this thread look like: img = load("fits/656nmos.fits", wcsdims=true)
implot(img) |
oop, think my finger slipped, sorry! |
Ideas/Wishlist for AstroImages
As discussed on the JuliaAstro call, I'm interested in developing out AstroImages further. The following is a sort of wishlist of what I would want in such a package. I'm aware several of these features like plot recipes already exist in this package, but I've included them below anyways for completeness.
The purpose of all of this is just to start a discussion and summarize what the community would want.
In terms of previous work in this space, there are to my knowledge:
Basics
Array types
Integration with Images.jl ecosystem
FITS Headers
img.totexp=5
. This is really convenient but we'll still need another syntax for accessing header comments.img[<:Union{Number,Vector}]
for indexing into the imageimg[:head1]
for accessing a header valueimg[:head1, /]
for accessing a comment (but this may be too punny)img.head1
as a shortcut syntax for accessing a header value.I implemented this syntax in DirectImages after the discussion we had at Compact syntax for getting, setting header values and comments FITSIO.jl#128 and it has felt very natural. Feedback welcome though.
Axes and WCS
Displaying Images
imshow
that displays images with more customization, e.g. colorschemes, clipping. I prototyped this and found displaying a large image with full color this way was >10x faster than through plots. E.g. 1600x1600 takes about 150ms.plot()
ing images of course.plot()
an array ofGray
for example than making a big heatmap. Hopefully we could leverage such animshow
function to make plotting large images efficient automatically.plot
should be WCS aware vsimshow
that just shows the pixel data.External Image Viewing Integration
ds9show
from DirectImages.jlThe text was updated successfully, but these errors were encountered: