KAT-1196: Support planets and other non-radec targets in mvftoms #333

ludwigschwardt · 2021-04-21T09:45:08Z

It is time...

@mauch, I used the script you modified for @nirujmohanr as basis, and also threw in scans for good measure.

Very early on (see 73546ac) I decided to disable non-radec targets because I did not understand the MS format well enough to be sure that it would work. The FIELD subtable stores directions that may vary as polynomials over time and it all seemed very complicated. Fast-forward 10 years to the Great Conjunction and a subsequent clamour for MSes with Jupiter... Having studied the `fixplanets` CASA task and the `planets_6cm` demo dataset, I came to the conclusion that it is much ado about nothing. The standard procedure in CASA picks the first timestamp associated with a field and determines the (ra, dec) from the (az, el) POINTING table in the MS. In other words, just like `target.radec(first_timestamp)` :-) Looking at `fixplanets`, there is also an option to use the Horizons ephemeris as an alternative. If we figure this out someday it might be more accurate for Solar System objects, but for now this trick works for all non-radec targets. 🎉

The CASA `fixplanets` task reveals that the DIRECTION column in the POINTING subtable of an MS contains (az, el) coordinates in radians. Not that we are using it yet, but just putting it out there.

While we are at it, also allow scans (and not only tracks) into an MS. The FIELD and SOURCE tables will be populated from the target (ra, dec) and the only difference is that the antenna pointing won't match that. But we are not storing the POINTING table yet, so it should not make a difference. This at least provides users with the ability to look at scan data and figure it out themselves :-)

mauch

Just a minor question about the update to the POINTING table docstring, but otherwise looks good.

Great to see a KAT-7 JIRA ticket getting some attention :)

mauch · 2021-04-21T11:23:15Z

katdal/ms_extra.py

@@ -863,7 +863,7 @@ def populate_pointing_dict(num_antennas, observation_duration, start_time, phase
    start_time : float
        Start time of observation, as a Modified Julian Date in seconds
    phase_center : array of float, shape (2,)
-        Direction of phase center, in ra-dec coordinates as 2-element array
+        Direction of phase center, in (az, el) coordinates as 2-element array (?)


What does the (?) signify here?

I'm still not sure... Ideally this should be an array of shape e.g. (num_times, 2, num_antennas) but a single phase center for all antennas and all times is definitely not it.

spassmoor · 2021-04-21T11:47:05Z

scripts/mvftoms.py

                    field_names.append(target.name)
+                    # Set direction of the field center to the (ra, dec) at the start
+                    # of the first valid scan, based on the reference (catalogue) antenna.
+                    field_time = katpoint.Timestamp(utc_seconds[0])


This is very similar to the hack that I did in 2013 when I image Jupiter. I used one ra,dec coordinate for the full observation. Does this give one coordinate pair per track or one coordinate pair per observation ?

I also noted star trails in the automated reduction that uses katdal->obit . How are they doing it @mauch ?

In katsdpcontim the target ra,dec is derived (for all observation time) from the the first timestamp of the observation.

Yes Sean, I was pleased to see that in KAT-1196 after I rediscovered the ticket. I'm also glad you didn't close it, which provided me with a ready-to-use ticket :-)

This gives one coordinate pair per observation. It doesn't use the first timestamp of the entire observation, but the first timestamp of a track on the actual source, mirroring CASA.

I tried the option of one coordinate pair per track, but that would imply the correspondence:

MeerKAT track <-> CASA field

The reality is the following correspondence:

MeerKAT track <-> CASA scan MeerKAT target <-> CASA field / source

So you are stuck with a single (ra, dec) per target.

This "hack" is also apparently used by CASA in the fixplanets task. There you have 3 options:

Use the POINTING (az, el) table to get (ra, dec) for either the first timestamp of the field (the default option) or the median unflagged timestamp, using a given reference antenna (defaults to antenna 0).

Use an ephemeris, but this is a bit obscure. It gets added to an EPHEMERIS table while the FIELD (ra, dec) = (0, 0).

Override the field direction explicitly.

It seems that the three field centers don't really matter, unless you plan to run fixvis to change the UVW. As long as the delay tracking did the right thing for your source, CASA is happy.

Or maybe the field centers are used to determine the absolute coordinates of the imaging grid?

OK, so I'm just slightly confused. I think I understand, but please confirm that I do:

I see one RA/Dec entry for Jupiter in FIELD. So this is position of Jupiter at at the start of the first scan on it, correct? Thereafter, the correlator delay (and UVWs, etc.) is tracking Jupiter, correct? So it's a "kludge" in the sense that only the first timeslot's delay is really towards the given RA/Dec of its stated field.

I should therefore expect the imager to give me a proper image of Jupiter, and smudged tracks for the background sources, correct?

In katdal you could try:

d = katdal.open(...) d.select(...) d.catalogue['Jupiter'].radec(d.timestamps) # radec at the selected timestamps

This does not consider target offsets in the delay model but those would not normally be used. And it assumes you did not change the ref_ant from its nominal value of 'array' when you opened the dataset.

Mesmerising 🤩

The basic parameters look good:

Jupiter was 5.09 AU from us at the Conjunction and with 140 000 km diameter you get an extent of ~38" ✅

Jupiter's (ra, dec) changed by ~210" in 6.5 hours, which is about the length of the star trails ✅ Unless that was a 9-10 hour observation like the ones in the archive, then they are a bit short.

I think that's Jupiter :-)

And now, the movie version! http://ratt.radiopadre.net/~oms/jove.gif. This is about 4 hours' worth.

I was going to ask whether there would be any parallactic rotation of Jupiter's radio axis with respect to (ra, dec) and now I know, thanks to the movie :-D

And the spacing between Jupiter and Saturn also seems right for the conjunction (~0.1º).

I like the radio galaxy photobombing this auspicious moment 😂

o-smirnov · 2021-04-21T13:04:12Z

Thanks @ludwigschwardt, I started the download and I can see the Jupiter scans arriving now. 👍

mauch · 2021-04-23T10:37:27Z

Ooh! Thats cool. Is that Saturn to the North moving slowly WRT the background?

…

On Fri, Apr 23, 2021 at 12:16 PM Oleg Smirnov ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In scripts/mvftoms.py <#333 (comment)>: > field_names.append(target.name) + # Set direction of the field center to the (ra, dec) at the start + # of the first valid scan, based on the reference (catalogue) antenna. + field_time = katpoint.Timestamp(utc_seconds[0]) And now, the movie version! http://ratt.radiopadre.net/~oms/jove.gif. This is about 4 hours' worth. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#333 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABIRDD3UL6KSOV34KZ6TPJDTKFCIZANCNFSM43J73XFA> .

o-smirnov · 2021-04-23T10:39:48Z

Yep, it can only be Saturn. This is the "conjunction" track.

o-smirnov · 2021-05-28T22:36:05Z

Every good movie deserves a sequel, and I gotta say, this is one case of the sequel blowing away the original: http://ratt.radiopadre.net/~oms/jove3b.png

ludwigschwardt · 2021-05-28T22:53:14Z

"I'll be back..." Wow, two photobombers! Awesome stuff.

o-smirnov · 2021-05-29T09:45:17Z

Look carefully, you might spot a transient going pop near one of them. And the steady little dot in the centre of the upper left inset is Callisto.

Is there a way to get the actual per-integration delay centres out of katdal? I'm trying to figure out if the shenanigans we engaged in with coordinates in the MS have introduced a small systematic offset into the astrometry...

o-smirnov · 2021-05-29T11:46:06Z

I'm trying to figure out if the shenanigans we engaged in with coordinates in the MS have introduced a small systematic offset into the astrometry...

Actually, I think we're all good. I checked against an ephemeris calculator, and Saturn is spot-on.

bennahugo · 2021-05-29T15:14:20Z

You can query elev and az directly from katdal?

…

On Sat, 29 May 2021, 13:46 Oleg Smirnov, ***@***.***> wrote: I'm trying to figure out if the shenanigans we engaged in with coordinates in the MS have introduced a small systematic offset into the astrometry... Actually, I think we're all good. I checked against an ephemeris calculator, and Saturn is spot-on. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#333 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4RE6SB334QXJA7ATXHO2LTQDHYVANCNFSM43J73XFA> .

o-smirnov · 2021-05-29T20:01:38Z

I would need to be taught. But not this time. The ephemeris checks out, the astrometry is spot-on. Onwards and upwards.

ludwigschwardt added 3 commits April 21, 2021 11:05

KAT-1196 Update POINTING table docstring

30382f6

The CASA `fixplanets` task reveals that the DIRECTION column in the POINTING subtable of an MS contains (az, el) coordinates in radians. Not that we are using it yet, but just putting it out there.

ludwigschwardt requested review from mauch and spassmoor April 21, 2021 09:45

mauch approved these changes Apr 21, 2021

View reviewed changes

spassmoor approved these changes Apr 21, 2021

View reviewed changes

ludwigschwardt merged commit 32d139f into master Apr 21, 2021

ludwigschwardt deleted the kat-1196-support-planets-in-mvftoms branch April 21, 2021 13:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KAT-1196: Support planets and other non-radec targets in mvftoms #333

KAT-1196: Support planets and other non-radec targets in mvftoms #333

ludwigschwardt commented Apr 21, 2021

mauch left a comment

mauch Apr 21, 2021

ludwigschwardt Apr 21, 2021

spassmoor Apr 21, 2021

mauch Apr 21, 2021

ludwigschwardt Apr 21, 2021

ludwigschwardt Apr 21, 2021

o-smirnov Apr 21, 2021

ludwigschwardt Apr 22, 2021 •

edited

Loading

o-smirnov Apr 22, 2021

ludwigschwardt Apr 22, 2021 •

edited

Loading

o-smirnov Apr 23, 2021

ludwigschwardt Apr 23, 2021

o-smirnov commented Apr 21, 2021

mauch commented Apr 23, 2021 via email

o-smirnov commented Apr 23, 2021

o-smirnov commented May 28, 2021

ludwigschwardt commented May 28, 2021 •

edited

Loading

o-smirnov commented May 29, 2021

o-smirnov commented May 29, 2021

bennahugo commented May 29, 2021 via email

o-smirnov commented May 29, 2021

KAT-1196: Support planets and other non-radec targets in mvftoms #333

KAT-1196: Support planets and other non-radec targets in mvftoms #333

Conversation

ludwigschwardt commented Apr 21, 2021

mauch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ludwigschwardt Apr 22, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ludwigschwardt Apr 22, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

o-smirnov commented Apr 21, 2021

mauch commented Apr 23, 2021 via email

o-smirnov commented Apr 23, 2021

o-smirnov commented May 28, 2021

ludwigschwardt commented May 28, 2021 • edited Loading

o-smirnov commented May 29, 2021

o-smirnov commented May 29, 2021

bennahugo commented May 29, 2021 via email

o-smirnov commented May 29, 2021

ludwigschwardt Apr 22, 2021 •

edited

Loading

ludwigschwardt Apr 22, 2021 •

edited

Loading

ludwigschwardt commented May 28, 2021 •

edited

Loading