Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-40918: Use unified CCS/OCS sequence number for exposure_id #472

Merged
merged 2 commits into from
Sep 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 8 additions & 1 deletion python/lsst/obs/lsst/translators/lsst.py
Original file line number Diff line number Diff line change
Expand Up @@ -412,13 +412,20 @@ def compute_exposure_id(dayobs, seqnum, controller=None):
if seqnum >= 10**_SEQNUM_MAXDIGITS:
raise ValueError(f"Sequence number ({seqnum}) exceeds limit")

dayobs = int(dayobs)
if dayobs > 20231004 and controller == "C":
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ktlim this is the line that needs to change when we decide on a date. Ideally this would be in a weekly that can be used by the ingest system (and OODS? cc/ @srp3rd ) before the switch over date. We can instead pick a date in November here, let camera change to unified numbering next week, and just allow a month of data to use the 1000 offset -- there won't be any clashes and it will be fine, just odd looking for people (but it's already odd for people now)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be nice to move the cast up above the seqnum code to where the rest of the dayobs handling is. But the if pertains to controller so should probably stay here.

Yes, the OODS also uses this ingest code. I understand that it can be any weekly (although I think OODS and auto-ingest can use dailies if need be) after the Camera has actually switched, but the fewer images that are taken in between the better. (I suspect that most of the controller-C images taken in between will actually be tests of the system.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tony-johnson is 20231004 a reasonable date for me to pick if I merge by tomorrow (so would be in this weekly)?

# As of this date the CCS controller has a unified counter
# with the OCS, so there is no need to adjust the dayobs
# to make unique exposure IDs.
controller = None

# Camera control changes the exposure ID
if controller is not None:
index = CONTROLLERS.find(controller)
if index == -1:
raise ValueError(f"Supplied controller, '{controller}' is not "
f"in supported list: {CONTROLLERS}")
dayobs = int(dayobs)

# Increment a thousand years per controller
dayobs += _CONTROLLER_INCREMENT * index

Expand Down
142 changes: 142 additions & 0 deletions tests/headers/lsstCam-faked-future-ccs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
SIMPLE: true
BITPIX: 32
NAXIS: 2
EXTEND: true
DATE: '2019-03-19T15:50:34.622'
DATE-OBS: '2019-03-19T15:50:28.145'
MJD: 58561.66012293985
MJD-OBS: 58561.66004797444
IMAGETAG: '1553010628145'
TSTAND: BOT
INSTRUME: lsstCam
CONTNUM: 0000189126e1
CCD_MANU: ITL
CCD_TYPE: 3800C
CCDSLOT: S02
RAFTBAY: R10
LSST_NUM: ITL-3800C-041
RAFTNAME: LCA-11021_RTM-003_ETU2
TESTTYPE: BIAS
IMGTYPE: BIAS
SEQNUM: 1
DETSIZE: '[1:4072,1:4000]'
CCDTEMP: -110.09798421488179
EXPTIME: 0.0
FILENAME: MC_C_20190319_000001_R10_S02.fits
BINX: 1
BINY: 1
HEADVER: 1
CCDGAIN: 1.0
CCDNOISE: 10.0
TELESCOP: LSST
OBSID: MC_C_20190319_000001
TELCODE: MC
CONTRLLR: C
DAYOBS: '20290319'
TSEQNUM: 0
TEMPLED1: 20.600147112211552
TEMPLED2: 20.18183822135534
TEMPSPH: 24.385273966899142
TEMPBRD: 29.7677403380139
CHECKSUM: Y9fAb7c8Z7cAa7c7
DATASUM: '139019781'
XTENSION: IMAGE
NAXIS1: 576
NAXIS2: 2048
PCOUNT: 0
GCOUNT: 1
CHANNEL: 1
EXTNAME: Segment10
CCDSUM: 1 1
AVERAGE: 25604.065213846083
AVGBIAS: 25604.041137695312
STDVBIAS: 2.0006144295067294
STDEV: 2.007842126768741
DATASEC: '[4:512,1:2000]'
DETSEC: '[509:1,1:2000]'
DTV1: 513
DTV2: 0
DTM1_1: -1.0
DTM2_2: 1.0
DTM1_2: 0.0
DTM2_1: 0.0
WCSNAMEA: AMPLIFIER
CTYPE1A: Seg_X
CTYPE2A: Seg_Y
PC1_1A: 0.0
PC1_2A: -1.0
PC2_1A: -1.0
PC2_2A: 0.0
CDELT1A: 1.0
CDELT2A: 1.0
CRPIX1A: 0.0
CRPIX2A: 0.0
CRVAL1A: 2001.0
CRVAL2A: 507.0
WCSNAMEC: CCD
CTYPE1C: CCD_X
CTYPE2C: CCD_Y
PC1_1C: 0.0
PC1_2C: -1.0
PC2_1C: -1.0
PC2_2C: 0.0
CDELT1C: 1.0
CDELT2C: 1.0
CRPIX1C: 0.0
CRPIX2C: 0.0
CRVAL1C: 4001.0
CRVAL2C: 507.0
WCSNAMER: RAFT
CTYPE1R: RAFT_X
CTYPE2R: RAFT_Y
PC1_1R: 0.0
PC1_2R: -1.0
PC2_1R: -1.0
PC2_2R: 0.0
CDELT1R: 1.0
CDELT2R: 1.0
CRPIX1R: 0.0
CRPIX2R: 0.0
CRVAL1R: 4126.0
CRVAL2R: 9046.0
WCSNAMEF: FOCAL_PLANE
CTYPE1F: FP_X
CTYPE2F: FP_Y
PC1_1F: 0.0
PC1_2F: -1.0
PC2_1F: -1.0
PC2_2F: 0.0
CDELT1F: 1.0
CDELT2F: 1.0
CRPIX1F: 0.0
CRPIX2F: 0.0
CRVAL1F: 16826.0
CRVAL2F: 9046.0
WCSNAMEB: CCD_SERPAR
CTYPE1B: CCD_S
CTYPE2B: CCD_P
PC1_1B: -1.0
PC1_2B: 0.0
PC2_1B: 0.0
PC2_2B: -1.0
CDELT1B: 1.0
CDELT2B: 1.0
CRPIX1B: 0.0
CRPIX2B: 0.0
CRVAL1B: 507.0
CRVAL2B: 4001.0
WCSNAMEQ: RAFT_SERPAR
CTYPE1Q: RAFT_S
CTYPE2Q: RAFT_P
PC1_1Q: -1.0
PC1_2Q: 0.0
PC2_1Q: 0.0
PC2_2Q: -1.0
CDELT1Q: 1.0
CDELT2Q: 1.0
CRPIX1Q: 0.0
CRPIX2Q: 0.0
CRVAL1Q: 9046.0
CRVAL2Q: 4126.0
BSCALE: 1.0
BZERO: 0.0
29 changes: 29 additions & 0 deletions tests/test_translator.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,35 @@ def test_lsstCam_translator(self):
science_program="6549D",
temperature=None,
visit_id=3019040600643)),
("lsstCam-faked-future-ccs.yaml", # Old CCS observation with date in future for exp_id
dict(telescope="Simonyi Survey Telescope",
instrument="LSSTCam",
boresight_rotation_coord="unknown",
dark_time=0.0*u.s,
detector_exposure_id=0xdb480011d,
detector_group="R10",
detector_name="S02",
detector_num=29,
detector_serial="ITL-3800C-041",
exposure_id=2029031900001,
exposure_group="2029031900001",
exposure_time=0.0*u.s,
focus_z=0.0*u.mm,
group_counter_end=1,
group_counter_start=1,
has_simulated_content=False,
object="UNKNOWN",
observation_counter=1,
observation_id="MC_C_20190319_000001",
observation_type="bias",
observation_reason="bias",
observing_day=20290319,
physical_filter="unknown",
pressure=None,
relative_humidity=None,
science_program="unknown",
temperature=None,
visit_id=2029031900001)),
)
for filename, expected in test_data:
with self.subTest(f"Testing {filename}"):
Expand Down