NIRCam TS grism target offsets are not handled

[stscijgbot-jp]

Issue JP-3595 was created on JIRA by Howard Bushouse:

As documented in JSOCINT-756, the APT template for NIRCam time series grism observations allows the user to move the target away from the normal aperture reference location via an offset special requirement ("Offset SR"). The set_telescope_pointing routines, as well as all downstream calibration steps, blindly assume that the target is always located at the aperture reference point and hence will be incorrect when an Offset SR is in use. Need to update all points within the set_telescope_pointing functions and Cal steps to add the offsets to the assumed target location for this mode. An example proposal using an Offset SR has been run through the OTB and the data are available to DMS for examination, in order to determine what metadata can be used to obtain the offset values.

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

The XREF_SCI/YREF_SCI keywords in the FITS headers are currently populated by set_telescope_pointing based on the CRPIX1/CRPIX2 values retrieved from the SIAF, which are the nominal aperture reference point:

https://github.com/spacetelescope/jwst/blob/master/jwst/lib/set_telescope_pointing.py#L2072

It's possible that the XOFFSET/YOFFSET keywords in the FITS headers may contain the offsets that we need to apply to the nominal aperture reference point. This needs to be confirmed. Note that XOFFSET/YOFFSET are in units of arcsec.

https://github.com/spacetelescope/stdatamodels/blob/main/src/stdatamodels/jwst/datamodels/schemas/core.schema.yaml#L1298

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

For the 00024 test proposal, using observation 26, the APT file shows requested offsets of 20.0, -10.0 arcsec. The XOFFSET, YOFFSET values in the resulting FITS headers are 20.000059 and -8.5516085, respectively. So that's close to the requested offsets, but still off by ~1.5 arcsec in Y. At the assumed pixel scale of ~0.06"/pix for the long-wave channel of NIRCam, that's a whopping 24 pixels!

[stscijgbot-jp]

Comment by Nadia Dencheva on JIRA:

The XOFFSET/YOFFSET keywords are the dither offsets. They are in arcsec in the SIAF Ideal coordinate system. If they are reused by APT for the source offset then they need to be converted to x/y offset in the detector system. There are functions in the pipeline which do this. But yes, first we need to figure out if these are the keywords to use for this use case.

[stscijgbot-jp]

Comment by David Law on JIRA:

Jumping in since this involves coordinates: XOFFSET/YOFFSET keywords recorded in the FITS headers combine both the dither offset and any requested special requirement offset from the APT file.  Everything is in Ideal coordinates (for the aperture in use) arcsec throughout; the dither reference files in the PRD are in Ideal, special requirement offsets in APT are in Ideal, and the header keyword values combining the two are in Ideal.

[stscijgbot-jp]

Comment by Ned Molter on JIRA:

For observation 15 from PID 1185, an offset of X=-5.276", Y=0.0092" was requested.  In the fits header I am seeing:

        Dither information                                                      
                                                                                
PATTTYPE= 'NONE    '           / Primary dither pattern type                    
PATT_NUM=                    1 / Position number within dither pattern          
NUMDTHPT=                    1 / Total number of points in pattern              
XOFFSET =   -5.276074012789024 / x offset from pattern starting position        
YOFFSET =   1.4530882023150162 / y offset from pattern starting position  ```
The X offset is exactly as requested, whereas the Y offset is the same 1.45" away from what was requested as for the test proposal Howard looked at.  Any ideas where that offset is coming from?

[stscijgbot-jp]

Comment by Ned Molter on JIRA:

For observation 15 from PID 1185, an offset of X=-5.276", Y=0.0092" was requested.  In the fits header I am seeing:

        Dither information                                                      
                                                                                
PATTTYPE= 'NONE    '           / Primary dither pattern type                    
PATT_NUM=                    1 / Position number within dither pattern          
NUMDTHPT=                    1 / Total number of points in pattern              
XOFFSET =   -5.276074012789024 / x offset from pattern starting position        
YOFFSET =   1.4530882023150162 / y offset from pattern starting position  ```
The X offset is exactly as requested, whereas the Y offset is the same 1.45" away from what was requested as for the test proposal Howard looked at.  Any ideas where that offset is coming from?

[stscijgbot-jp]

Comment by David Law on JIRA:

Ned Molter That's interesting; offhand I don't know what's going on there, although there are a couple of possibilities that come to mind.  I'm following up with a few folks to see if we can get a better answer.  Just to clarify, I see the same offset in the APT pointings file for that program, so it's something coming in at the APT stage.

[stscijgbot-jp]

Comment by Ned Molter on JIRA:

David Law got a response from Mario Gennaro already on the related JSOCINT ticket.  I'm not sure who is in charge of making the required update to the definition of that keyword, though.

[stscijgbot-jp]

Comment by Ned Molter on JIRA:

I think I have this working locally now.  I attached a figure to this ticket.  I spent a long time trying to validate the data against a model star spectrum (black line) at a similar Teff, log_g, and metallicity, but the spectral lines weren't similar enough between model and data.

Thankfully, the program whose test data I was using observed GJ 436 both with (orange line) and without (blue line) an x-offset specified.  The figure shows that the spectral lines overlap nicely, while the wavelength where good data begins (indicative of pixel location on the detector) is different.

The fix required two changes, first to adjust the reference pixel, and then to make the grid over which extract_2d.grisms.compute_wavelength_array() honor that offset. There is one small bookkeeping issue, which is that extract_2d needs to know how large the offset was in pixels.  In my local code, I re-used the slit.meta.dither.x_offset keyword, which originally had the value in arcsec.  Of course this isn't the permanent way of doing this, but how should this be handled?  I see two options:

• make a new keyword in either dither or wcs that contains the value
• compute the value in pixels from the value in arcsec using the wcs transforms twice, once in assign_wcs and once in extract_2d

There may be another way of doing this, but it's not obvious to me - any ideas are welcome!

[stscijgbot-jp]

Comment by Ned Molter on JIRA:

It occurs to me that I can just store the slit.xcentroid and slit.ycentroid, consistent with grism observations that contain multiple sources, and then subtract xref_sci, yref_sci from that to get the offset later.  I'm going to go ahead with this unless someone else has a strong opinion

This doesn't work because the data are of course not a slitmodel yet during assign_wcs().

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

Fixed by #8449