Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General clean-ups and speed-ups for DR9 work #658

Merged
merged 12 commits into from Dec 5, 2020
Merged

Conversation

geordie666
Copy link
Contributor

This PR cleans-up (and speeds-up) some functionality as we move towards producing DR9 targets and fiberassign files. It includes:

  • A corrected data model when repartitioning skies into their correct HEALPixels.
    • skies are run in bricks that are centered in HEALPixels, rather than in the actual boundaries of a HEALPixel, so they need to be "repartitioned" into the correct pixels (see Repartition sky files so sky locations lie within HEALPixel boundaries #621); this PR changes the organization of the input/output files to automatically reflect the data model that I have started to use when releasing targets.
  • Faster versions of all of the desitarget.io.read_targets_in_X functions:
    • i.e., read_targets_in_tiles(quick=True), read_targets_in_box(quick=True), read_targets_in_cap(quick=True), read_targets_in_hp(quick=True).
    • The new functions are less general, but they run 2-10x faster.
    • The speed-up is achieved by assuming the HEALPixel-split data model rather than by checking and determining the data model from header information (so the "quick=True" versions are more fragile).
    • The speed-up is greater for HEALPixel-split files at higher nside. Comparing targets partitioned into files at nside=2 versus nside=8, the speed-up was 5x.
  • Establishing "standard" MASKBITS combinations for all of the target classes that can be read via convenience functions, and using those convenience functions to automatically inform the "FRACAREA_X" calculations in pixweight files.
    • This makes it far easier to ensure that the MASKBITS cuts adopted by the target files are always aligned with those calculated in the pixweight files.
  • Catching the case where a user tried to construct MTL ledgers at a lower resolution than the corresponding target files (which partially addresses Ledger code failure for high to low res, e.g. with mock targets. #650).

@geordie666 geordie666 mentioned this pull request Dec 5, 2020
@geordie666 geordie666 merged commit c6959b1 into master Dec 5, 2020
@geordie666 geordie666 deleted the ADM-speedup-cleanup branch December 5, 2020 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant