Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

generate tilepix.fits and tilepix.json files from reduced data products #1614

Merged
merged 7 commits into from Jan 21, 2022

Conversation

moustakas
Copy link
Member

Addresses #1374 by reading the FIBERMAP tables contained in the healpix-organized spectra- files rather than in the preproc directory, thereby guaranteeing that the tilepix.fits and tilepix.json files will only contain healpixels which have fully reduced spectra on-disk (for a given data release).

I've run a few small tests and verified that it addresses the examples given in #1374. However, I'm running a full-Everest test right now with

desi_map_tilepix --reduxdir /global/cfs/cdirs/desi/spectro/redux/everest --nside 64 -o new-tilepix.fits

and will report back when that is complete.

Note: I'm not sure where/when desi_map_tilepix is currently invoked in the pipeline processing steps, but if this PR is accepted then it means that it will have to be run at the end of a data release after the healpix directory has been fully populated.

@moustakas moustakas changed the title generate tilepix.fits and tilepix.json files from fully reduced data products; fix #1374 generate tilepix.fits and tilepix.json files from fully reduced data products Jan 21, 2022
@moustakas moustakas changed the title generate tilepix.fits and tilepix.json files from fully reduced data products generate tilepix.fits and tilepix.json files from reduced data products Jan 21, 2022
@coveralls
Copy link

coveralls commented Jan 21, 2022

Coverage Status

Coverage remained the same at 25.201% when pulling f202c1e on fix-tilepix-map into ea76485 on master.

@moustakas
Copy link
Member Author

It's possible this PR also addresses #1379, although I'm a little unclear about this statement from @sbailey because this PR only affects the tilepix.json and tilepix.fits files, not the tiles-{specprod}.fits file:

These [dither] test tiles shouldn't be in tilepix.json, and tilepix.json and tiles-{specprod}.fits should have the same tiles, and all of those should match what is actually on disk in healpix/{survey}/{faprgrm}/

It's possible (quite likely!) that I'm just muddying the waters here and, if so, apologies.

@moustakas
Copy link
Member Author

Also fixes #1373.

@sbailey
Copy link
Contributor

sbailey commented Jan 21, 2022

Thanks. I expect we'll make a few more last-minute changes to the healpix workflow after Fuji is underway, but healpix has to wait for all tile-based processing to be finished first so it doesn't have to be a blocking factor. I'll merge now to get it in the first Fuji tag, and if we need to make more updates later ok.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants