Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review for DM-2674: Make meas_mosaic work on LSST. #1

Merged
merged 44 commits into from
Jul 5, 2016

Commits on Feb 15, 2016

  1. Use LSST exception names.

    jdswinbank committed Feb 15, 2016
    Configuration menu
    Copy the full SHA
    8c319f9 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    70f1d62 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    53fc296 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    a76fac9 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2016

  1. Configuration menu
    Copy the full SHA
    6cc9af5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    189183b View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2016

  1. Adapt to new meas_astrom.

    jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    751b6ff View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e31fc2c View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    c069ccb View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    c700ef9 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    da21d15 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    f36d3fd View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    33a48ea View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    a32f6bf View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    6994ed3 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    98984e4 View commit details
    Browse the repository at this point in the history
  11. Adapt to running on OSX/SIP.

    * Export LSST_LIBRARY_PATH.
    * Rewrite shebang lines on files in bin.src.
    jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    1c27fa9 View commit details
    Browse the repository at this point in the history
  12. Correct include file name.

    jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    94f3c00 View commit details
    Browse the repository at this point in the history
  13. Reword source loading for LSST.

    LSST does not persist an icMatchFull equivalent, so we need to construct this
    on the fly based on the source and match records.
    jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    a627301 View commit details
    Browse the repository at this point in the history
  14. Make it possible to correct flux only.

    If MosaicTask runs with doSolveWcs=False, it does not write a new Wcs. It
    should still be possible to apply the flux correction in this case.
    jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    f961819 View commit details
    Browse the repository at this point in the history
  15. Specify coadd type in MosaicTaskConfig.

    This is required by the LSST implementation of PerTractCcdDataIdContainer.
    jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    a644076 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    8c67389 View commit details
    Browse the repository at this point in the history
  17. Make it possible to correct Wcs only.

    If MosaicTask runs with doSolveFlux=False, it does not write a new flux
    correction. It should still be possible to apply the new Wcs in this case.
    jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    a08f7d2 View commit details
    Browse the repository at this point in the history
  18. Update to match recent Butler API changes.

    This is necessary following the merger of DM-4683.
    jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    1ce11a3 View commit details
    Browse the repository at this point in the history
  19. Update to match new ProcessCcdTask.

    The new (post-DM-4692) ProcessCcdTask uses the src catalog to perform
    astrometry and photometry (as opposed to icSrc). It also no longer saves
    icMatch, or attaches positional information to icSrcs.
    
    Here, we switch to using src in meas_mosaic.
    jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    6907577 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    626bf5d View commit details
    Browse the repository at this point in the history
  21. Adjust formatting & clean up.

    laurenam authored and jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    2e2038b View commit details
    Browse the repository at this point in the history
  22. De-boostification

    This is in response to DM-4036, DM-5879 and DM-6095.
    
    Includes:
    
    Migration to std::shared_ptr, including:
    - boost::shared_ptr -> std::shared_ptr
    - boost::weak_ptr -> std::weak_ptr
    - boost::static_pointer_cast -> std::static_pointer_cast
    - boost::dynamic_pointer_cast -> std::dynamic_pointer_cast
    - boost::const_pointer_cast -> std::const_pointer_cast
    - boost::reinterpret_pointer_cast -> std::reinterpret_pointer_cast
    - boost::enamble_shared_from_this -> std::enable_shared_from_this
    - boost pointer related includes -> standard library memory
    Replace boost::math.
    Replace boost::ref.
    laurenam authored and jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    3ef9dcb View commit details
    Browse the repository at this point in the history
  23. Add function name to print statements

    This is useful for debugging.
    laurenam authored and jdswinbank committed Jun 30, 2016
    Configuration menu
    Copy the full SHA
    d5a7939 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2016

  1. Account for catalogs read in janskys: fluxSigmas

    Also adds the appropriate fluxSigma key and field from which
    meas_mosaic extracts it.  This error has been propogated
    according to the colorterm correction.
    laurenam committed Jul 1, 2016
    Configuration menu
    Copy the full SHA
    f6b1ce5 View commit details
    Browse the repository at this point in the history
  2. Update diagnostics output.

    - Add labels and column headers.
    - Tweak diagnostic plots to include handling of CCD subsets, and labels and column headers.
    laurenam committed Jul 1, 2016
    Configuration menu
    Copy the full SHA
    056f48e View commit details
    Browse the repository at this point in the history
  3. Rotate catalog centroids to meas_mosaic coords

    If the detector has nQuarter%4 != 0 (i.e. it is rotated w.r.t the focal
    plane coordinate system), the (x, y) pixel values of the centroid slot for
    the source catalogs need to be rotated such that pixel (0, 0) corresponds
    to the LLC (i.e. the coordinate system expected by meas_mosaic).
    laurenam committed Jul 1, 2016
    Configuration menu
    Copy the full SHA
    927de8c View commit details
    Browse the repository at this point in the history
  4. Account for w<-->h for oddly rotated CCDs

    For the oddly rotated CCDs, the width and height values must be
    swapped when setting the bounding box for the spacial cell search.
    They also need to be swapped when drawing the CCD outlines on the
    diagnostic focal plane plots.
    laurenam committed Jul 1, 2016
    Configuration menu
    Copy the full SHA
    6bc2ac5 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2016

  1. Accommodate running on HSC stack products

    The HSC team is currently using a fork of the main LSST stack.  Many
    developements have occurred on this fork for the purpose of HSC SSP
    data release production runs.  What is currently the "HSC stack" will
    eventually become obsolete, once all new development that happened
    there is ported over to the LSST stack and validated.  For this
    validation, it is useful to be able to directly LSST vs. HSC stack
    runs when using the exact same inputs.  This is implemented here by
    including accommodations required to parse HSC stack produced outputs,
    namely a mapper between the different source schemas.  See the
    readSrc() function in SourceReader class in MosaicTask.py along with
    the config/hsc/Mosaic.py file in obs_subaru for an example of how to
    apply this mapper.  The mapper itself is in config/hsc/srcSchemaMap.py
    in obs_subaru.
    
    Note that these inputs will already be in the coordinate system that
    meas_mosaic expects (i.e. all CCDs have LLC pixel (0, 0) w.r.t the
    focal plane coordinate system), so no accommodations need to be made
    in that respect.
    laurenam committed Jul 5, 2016
    Configuration menu
    Copy the full SHA
    59d2d7d View commit details
    Browse the repository at this point in the history
  2. Add options to select against objects for mosaic fit

    Certain objects are not good candidates for inclusion in the mosaic
    fit.  Here we add optional selection criteria as config parameters
    to exclude objects based on values in the schema.  The following lists
    the config parameters.  If they are set to any valid key name, the
    behavior indicated is followed:
    
    extendednessForStarSelection: only include point-like sources
    saturatedForStarSelection: do not include saturated sources (unless
                               includeSaturated is True)
    psfStarForStarSelection: only include objects used in the PSF determination
                             *unless* extendednessForStarSelection, in which case
                             include all point-like sources
    calibStarForStarSelection: only include objects detected as calibration sources
    parentForStarSelection: do not include objects with parents
    nChildForStarSelection: do not include objects with children
    
    To exclude any given criterion, set its corresponding config value to an
    invalid value (e.g. None).
    laurenam committed Jul 5, 2016
    Configuration menu
    Copy the full SHA
    47a3aba View commit details
    Browse the repository at this point in the history
  3. Use full pivoting when using Eigen

    The matrix solvers will be operating on non-invertible matrices
    when input is sparse (e.g. operating on a limited number of inputs).
    Use full pivoting when using Eigen for a more robust solution.
    
    N.B. This may not be the optimal solution.  Will revisit if it is
    deemed necessary.
    laurenam committed Jul 5, 2016
    Configuration menu
    Copy the full SHA
    72d414f View commit details
    Browse the repository at this point in the history
  4. Force use of Eigen over MKL

    HSC is using Eigen, so while we are comparing between stacks, keep
    this consistent by disabling MKL on LSST stack for now.
    laurenam committed Jul 5, 2016
    Configuration menu
    Copy the full SHA
    6fe95ec View commit details
    Browse the repository at this point in the history
  5. Move appropriate functions to utils.py.

    These include the various diagnostic plotting and data output
    functions, functions accommodating HSC stack run analysis, and
    those used to rotate catalog pixel coordinate to accommodate the
    coordinate system expected by meas_mosaic (i.e. the LLC of any
    CCD as placed in the focal plane is pixel 0, 0).
    laurenam committed Jul 5, 2016
    Configuration menu
    Copy the full SHA
    536471f View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    ea73947 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    1a1a0cd View commit details
    Browse the repository at this point in the history
  8. Update applyMosaicResultsCatalog

    Updates the application of the results to a catalog.  Since meas_mosaic's
    solution is based on a coordinate system where a given detector's origin
    (pixel 0, 0) is associated with its LLC w.r.t. to the focal plane.  LSST
    always associates the detector origin with the electronics, thus if the
    detector was rotated when inserted into the focal plane (nQuarter%4 != 0),
    the centroids must be rotated to the meas_mosaic coordinate system before
    applying the corrections.
    
    Note that if working with catalogs produced by the HSC stack (hscRun is
    not None), the pixels are already in the correct coordinate system, so
    no rotations are necessary.
    laurenam committed Jul 5, 2016
    Configuration menu
    Copy the full SHA
    009cb28 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    f86d34b View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    3804a84 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    d5f1045 View commit details
    Browse the repository at this point in the history