Support complex image types

[pford]

Currently, all images are opened with float data type. Carta should support:

• Complex image
• Integer image
• Boolean Image

See #868

This issue separated to prioritise complex image types.

Issue separated from #388 , see also #77

For a complex-valued image, we need to support the following raster data requested from the frontend:

• real part of the complex value
• imaginary part of the complex value
• amplitude of the complex value (ie the Euler form)
• phase of the complex value (ie the Euler form)

[pford]

Copying comment from #388:
We should probably investigate alternative data delivery mechanisms for integer and boolean images. For integers, the byte-shuffling and Zstd compression approach taken with contours would probably work nicely. For booleans, sending them as bit arrays rather than byte arrays, and using Zstd (or even just the gzip compression built into WebSocket transmission) should be sufficient.

Converting to floating point and doing lossy compression with ZFP on quantized data would be counter-productive.

[jott3077]

Sanjay would like to see the complex images as a high priority as it is essential for ARDG. Is there a chance that can be addressed first?

[sanbw]

Yes please. Support for complex-valued images with options to display them as real, imaginary, amplitude or phase is what we (ARDG) needs. Basically the kind of support that exists in the current casa viewer.

[sanbw]

An example complex image can be picked up from /home/dhruva2/disk4/sanjay/CFCaches/CFS_0_0_CF_5_30_1.im. The header of this image looks like the following:

INFO Image data type : Complex
INFO + Image name : CFS_0_0_CF_5_30_1.im
INFO + Object name :
INFO + Image type : PagedImage
INFO + Image quantity : Intensity
INFO + Pixel mask(s) : None
INFO + Region(s) : None
INFO
INFO + Spectral reference : LSRK
INFO + Velocity type : RADIO
INFO + Rest frequency : 2.98513e+09 Hz
INFO + Telescope : UNKNOWN
INFO + Observer : UNKNOWN
INFO + Date observation : UNKNOWN
INFO +
INFO + Axis Coord Type Name Proj Shape Tile Coord value at pixel Coord incr Units
INFO + --------------------------------------------------------------------------------------------
INFO + 0 0 Linear UU 1361 195 0 1536.00 -2.062648e+00 lambda
INFO + 1 0 Linear VV 1361 171 0 1536.00 2.062648e+00 lambda
INFO + 2 1 Stokes Stokes 1 1 RR
INFO + 3 2 Spectral Frequency 1 1 2.669e+09 0.00 1.930088460380e+09 Hz
INFO + Velocity 31748.8 0.00 -1.938359e+05 km/s
INFO
INFO + Attached miscellaneous Information :
INFO + MiscInfo : Xsupport: Int 32
INFO + MiscInfo : Ysupport: Int 32
INFO + MiscInfo : Sampling: Float 20
INFO + MiscInfo : ParallacticAngle: Double -99.1609
INFO + MiscInfo : MuellerElement: Int 15
INFO + MiscInfo : WValue: Double 120733
INFO + MiscInfo : WIncr: Double 0.0074545
INFO + MiscInfo : Name: String "CFS_0_0_CF_5_30_1.im"
INFO + MiscInfo : ConjFreq: Double 3.309e+09
INFO + MiscInfo : ConjPoln: Int 8
INFO + MiscInfo : TelescopeName: String "EVLA"
INFO + MiscInfo : BandName: String "EVLA_S"
INFO + MiscInfo : Diameter: Float 25
INFO + MiscInfo : OpCode: Bool 0

[veggiesaurus]

@jott3077 Could we effectively treat the different representations of these complex numbers (real, imag, amplitude, phase) in a similar manner to the different Stokes parameters? The real and imaginary parts would be directly stored in the data, right? And the amplitude and phase would b calculated from the stored values. I know @arusstaylor has requested something similar for Stokes cubes (#433), so it might make sense to implement these two requests simultaneously.

[jott3077]

yes, that sounds similar. In fact I just got a request to calculate pol intensity and angle from Stokes cubes, which I think is implemented in the Stokes analysis widget but not yet for single images. SO the calculation back and forth seems to be similar. I will confirm with Sanjay if the formats can be either Re/Im and Amp/Ph or if Re/Im is the only one. In the first case the calcuations would need to go both ways between the representations.

…

On May 11, 2020, at 2:35 AM, Angus Comrie ***@***.***> wrote: @jott3077 <https://github.com/jott3077> Could we effectively treat the different representations of these complex numbers (real, imag, amplitude, phase) in a similar manner to the different Stokes parameters? The real and imaginary parts would be directly stored in the data, right? And the amplitude and phase would b calculated from the stored values. I know @arusstaylor <https://github.com/arusstaylor> has requested something similar for Stokes cubes (#433 <#433>), so it might make sense to implement these two requests simultaneously. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#520 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQJ6I7GCQT6MIWCON64QHTRQ62ETANCNFSM4M2RICMQ>.

[jott3077]

Sanjay, are the image formats always Re-Im values, or are there also formats with Amp-Ph? In other words, do the calculations need to go both ways or just from Re-Im -> Amp->ph? Juergen

…

On May 8, 2020, at 3:04 PM, Sanjay Bhatnagar ***@***.***> wrote: Yes please. Support for complex-valued images with options to display them as real, imaginary, amplitude or phase is what we (ARDG) needs. Basically the kind of support that exists in the current casa viewer. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#520 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQJ6I464JHOTNNS52BKPS3RQRXXFANCNFSM4M2RICMQ>.

[sanbw]

I suppose you are talking of storage format? If so, then Re/Im is the only one. Each pixel is complex-valued, which basically means that with each pixel, there two floating point numbers associated (in single or double precision). I am assuming that CARTA is using the CASACore's ImageInterface. If so, that's templated and all you need to do is open the image as ImageInterface<Complex> or ImageInterface<DComplex> object. Unless the internal CARTA data structures are general enough, it should be relatively simple to handle complex images.

…

On Mon, May 11, 2020 at 6:21 PM jott3077 ***@***.***> wrote: yes, that sounds similar. In fact I just got a request to calculate pol intensity and angle from Stokes cubes, which I think is implemented in the Stokes analysis widget but not yet for single images. SO the calculation back and forth seems to be similar. I will confirm with Sanjay if the formats can be either Re/Im and Amp/Ph or if Re/Im is the only one. In the first case the calcuations would need to go both ways between the representations. > On May 11, 2020, at 2:35 AM, Angus Comrie ***@***.***> wrote: > > > @jott3077 <https://github.com/jott3077> Could we effectively treat the different representations of these complex numbers (real, imag, amplitude, phase) in a similar manner to the different Stokes parameters? The real and imaginary parts would be directly stored in the data, right? And the amplitude and phase would b calculated from the stored values. I know @arusstaylor <https://github.com/arusstaylor> has requested something similar for Stokes cubes (#433 < #433>), so it might make sense to implement these two requests simultaneously. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub < #520 (comment)>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AFQJ6I7GCQT6MIWCON64QHTRQ62ETANCNFSM4M2RICMQ >. > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#520 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADG5B3IQ26IWBQFW2MHVFF3RRCJBLANCNFSM4M2RICMQ> .[image: Web Bug from https://github.com/notifications/beacon/ADG5B3IVYPYLNFLLEKTNMCDRRCJBLA5CNFSM4M2RICM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEVP4XTA.gif] [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": " #520 (comment)", "url": " #520 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": " https://github.com" } } ]

-- संजय

[Kechil]

Note that this was originally support integer, complex and boolean image types but has been split to isolate complex types in order to male the v3-b1 release.

[pford]

Once the backend image-handling code is templatized instead of ImageInterface<float>, it should be able to handle any datatype. I do not know that splitting off two data types is saving any work. The interface needs to be changed for handling complex data, so at the same time the raster data message could indicate to the frontend that the returned data is amp/phase/real/imag/bool/int. But it is up to @jolopezl whether splitting this issue would be easier and faster.

[veggiesaurus]

I think it still makes sense to split then up because complex images are a special case. It's not just a matter of reading a different data type, you also need to treat the complex cube like a number of separate cubes (similar to a Stokes hypercube), so that the user can visualise amp/real/imag/phase similar to IQUV

[pford]

Here's the doc from the original issue, with a link to sample images:
https://docs.google.com/document/d/1qH118KxBtxX4xtLNrV18meScxciFCVyIc9kelZ0JCvo/edit

[kswang1029]

Based on the recent slack discussion, shall we try to re-use LEL to support loading either real or imaginary or amp or phase of a complex image via the file browser?

[kswang1029]

@jolopezl @Kechil @sanbw @veggiesaurus with the current dev backend and frontend, we can load a complex image via LEL. For example, with
AMPLITUDE("complex.image"),
PHASE("complex.image"),
REAL("complex.image"), and
IMAG("complex.image"),
we can load the amplitude or phase or real part or imaginary part of the complex image, respectively.

The current remaining issue is that the backend cannot generate file info and header to be displayed at the frontend. Once the file info is available, the frontend can have a new dropdown similar to what we do for a FITS image with multiple HDUs to include the four components as amplitude, phase, real, and imaginary. Then based on user selection at the frontend, the frontend makes a mock up LEL expression to load the selected component internally.

Do you think the proposed approach fits the requirements?

[jolopezl]

@jolopezl @Kechil @sanbw @veggiesaurus with the current dev backend and frontend, we can load a complex image via LEL. For example, with AMPLITUDE("complex.image"), PHASE("complex.image"), REAL("complex.image"), and IMAG("complex.image"), we can load the amplitude or phase or real part or imaginary part of the complex image, respectively.

Thanks for the suggestion of using LEL. I think it will help get the complex image support now with less code refactoring.

The current remaining issue is that the backend cannot generate file info and header to be displayed at the frontend. Once the file info is available, the frontend can have a new dropdown similar to what we do for a FITS image with multiple HDUs to include the four components as amplitude, phase, real, and imaginary. Then based on user selection at the frontend, the frontend makes a mock up LEL expression to load the selected component internally.

Do you think the proposed approach fits the requirements?

I think so. That provides complex image support, but we will need to reevaluate with some user testing.

This should be ready quickly, some changes add needed to get the FileInfo of complex images too.