For the SPITZER-YAAAR data format, should we split up the modules further?

[ciskakemper]

Maybe @pscicluna has some insight in this one.

When reading in SPITZER-YAAAR data formats in data.py, the data is divided into chunks, by module. The data has four modules, corresponding to SL1, SL2, LL1 and LL2, but not in that order. In fact, the order is:

module	wavelength	name
0	7.5 - 14.0	SL2
1	5.2 - 8.3	SL1
2	20.6 - 37.5	LL2
3	14.1 - 21.4	LL1

Currently, the data is divided into two chunks, SL and LL, using the module information in the data file (e.g. (0 or 1) and (2 or 3)), but in that case, the chunks internally are not ordered by wavelength. Should we perhaps divide the data in four chunks, and order the chunks by wavelength?

[ciskakemper]

Oh, right, I see. There is some overlap between SL1 and SL2, and the same for LL1 and LL2. Splitting them up completely would not result in an ordered table. So my question is, why do we divide this data into two chunks to begin with. Can we not apply sort on the entire table?

[pscicluna]

Good query, and the answer is related to the calibration of the spectra. As I understand it, the short and long modules are effectively independent spectra, taken at different times and with different calibration through different apertures, which means that the noise model for each module needs to be independent. As a result, ampere is able to improve the calibration of each module as part of the fit. On the other hand, the two orders of each module are taken at the same time and with the same calibration, so should be calibrated together and should work with the same noise model. If this picture is wrong or incomplete we should revise how we do this.

…

________________________________ From: Ciska Kemper ***@***.***> Sent: Thursday, May 20, 2021 8:31:32 AM To: ICSM/ampere ***@***.***> Cc: Peter Scicluna ***@***.***>; Assign ***@***.***> Subject: Re: [ICSM/ampere] For the SPITZER-YAAAR data format, should we split up the modules further? (#33) Oh, right, I see. There is some overlap between SL1 and SL2, and the same for LL1 and LL2. Splitting them up completely would not result in an ordered table. So my question is, why do we divide this data into two chunks to begin with. Can we not apply sort on the entire table? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub<#33 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAD4J6RCC6YWELWP3ZFQMXTTOS3FJANCNFSM45GHTSYQ>.

[ciskakemper]

This makes a lot of sense. SL1 and SL2 are indeed observed by projection on the same detector array at the same time, and the same for LL1 and LL2. You basically have two detector array images with the spectra projected on it. Thus SL and LL are independent measurements but the different orders within each module are not. I think we can close this now.