Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

more robust when fiberassign in earlier expid #1529

Merged
merged 1 commit into from Dec 14, 2021
Merged

more robust when fiberassign in earlier expid #1529

merged 1 commit into from Dec 14, 2021

Conversation

sbailey
Copy link
Contributor

@sbailey sbailey commented Dec 13, 2021

This PR fixes #1272 where the "look in previous exposures for the fiberassign file" could end up grabbing the wrong fiberassign file. I originally was just going to turn off this look-back behavior since it isn't needed in normal ops, but there may be special test cases of setting up on a tile and then doing some manual exposures that could result in spectrograph exposures without a matching fiberassign file in the same EXPID, so the updated logic is:

  • if the raw data has a TILEID keyword, insist that the fiberassign file matches that
  • if looking to previous exposures without knowing the TILEID, check the data TARGTRA,TARGTDEC vs. fiberassign TILERA,TILEDEC for whatever fiberassign file is found and only accept it if they match

It also corrects a case where the coordinates file is missing fiber location information, by correctly flagging the fibers as FIBERSTATUS MISSINGPOSITION.

Example case of previous code finding and using an incorrect fiberassign file (now generates a RuntimeError because it can't find a good fiberassign file):

assemble_fibermap --overwrite -o blat.fits -n 20210114 -e 72405

Example case of correctly walking back to find a fiberassign file, albeit with missing info in the coordinates file resulting in FIBERSTATUS flags:

assemble_fibermap --overwrite -o blat.fits -n 20210220 -e 77103

This PR also introduces a utility function desispec.io.util.checkgzip(filename) that will look for a filename with or without the .gz extension; this is handy for cases like fiberassign where sometimes it is gzipped or not. We may also use this in the future with the pipeline if infrequently used files get post-facto gzipped and we want the code to easily find the right file either way.

@sbailey sbailey added this to In progress in Fuji via automation Dec 13, 2021
@sbailey
Copy link
Contributor Author

sbailey commented Dec 13, 2021

Note: the test coverage is better when run at NERSC, due to tests that require large/complex inputs and thus are not included in the github actions tests, but I have run unit tests of this branch directly at NERSC to include the assemble_fibermap tests.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.008%) to 25.682% when pulling f1d1166 on fibermap into 1273d08 on master.

@sbailey sbailey merged commit ac12047 into master Dec 14, 2021
Fuji automation moved this from In progress to Done Dec 14, 2021
@sbailey sbailey deleted the fibermap branch December 14, 2021 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Fuji
  
Done
Development

Successfully merging this pull request may close these issues.

pipeline using wrong fiberassign file if correct one wasn't transferred
2 participants