Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-245: Implement HSC camera in new camera framework #1

Merged
merged 81 commits into from
Feb 26, 2015

Commits on Feb 3, 2015

  1. Updates genDefectFits.py to use new CameraGeom

    Updates genDefectFits.py to use the new CameraGeom format.
    This also required a change in the hsc SConstruct file
    to match the new API.  Also repairs capitalizatio issue in
    scons file.
    Simon Krughoff authored and laurenam committed Feb 3, 2015
    Configuration menu
    Copy the full SHA
    f649b18 View commit details
    Browse the repository at this point in the history
  2. Wraps distortion code in XYTransform

    The new camera objects use XYTransforms to map from one
    coordinate system to another.  This means that the DistEst
    distortion code needs to be wrapped up in an XYTransform.
    
    The XYTransform is exposed in python by adding an entry to
    the XYTransformRegistry.
    Simon Krughoff authored and laurenam committed Feb 3, 2015
    1 Configuration menu
    Copy the full SHA
    9ee4c04 View commit details
    Browse the repository at this point in the history
  3. Adds a utility for building camera registries

    The old camera objects were built from policy files.  This
    is no longer required, but since the representation was available
    we continue to use the policy file as the starting point for
    building camera registries (the persisted files used by the mapper
    to build a camera).  Also, the transform registry is different
    between hsc and suprimecam.  This makes the repo generation
    respect this.
    Simon Krughoff authored and laurenam committed Feb 3, 2015
    6 Configuration menu
    Copy the full SHA
    181f7f9 View commit details
    Browse the repository at this point in the history
  4. Adds camera repository files

    This commit also updates the mapper policy files to
    point to the new camera repository.
    Simon Krughoff authored and laurenam committed Feb 3, 2015
    Configuration menu
    Copy the full SHA
    81251f8 View commit details
    Browse the repository at this point in the history
  5. Adds a new utility to display cameras

    This utility can display parts of the camera all the way to
    the whole mosaic.  It currently only works with HSC.
    There is another utility called showCamera that overlaps
    with this utility.  It can probably be removed, or it
    should be updated to work with the new cameraGeom.
    Simon Krughoff authored and laurenam committed Feb 3, 2015
    Configuration menu
    Copy the full SHA
    b21e6ef View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2015

  1. hscMapper: removes std_camera

    The use of std_camera was re-defined in the current master
    CameraMapper.
    Simon Krughoff authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    ae9491a View commit details
    Browse the repository at this point in the history
  2. Removes white spaces from hscMapper.py

    Simon Krughoff authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    40655b3 View commit details
    Browse the repository at this point in the history
  3. config: remove root.measurement.algorithms references outside try sta…

    …tements in processCcd.py and hsc/processCcd.py
    
    Removed those measurements that are not defined on the LSST side.
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    95fe405 View commit details
    Browse the repository at this point in the history
  4. isr: rename overscanPolyOrder to overscanOrder

    As per a previous code review requesting overloading of "order" to
    mean either polynomial order or spline knots.
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    9fb1fac View commit details
    Browse the repository at this point in the history
  5. Remove whitespace

    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    e77d768 View commit details
    Browse the repository at this point in the history
  6. Remove casts of form afwCG.cast_Ccd

    Polymorphism of detector-like objects was removed, so casts are
    not necessary.
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    201dcd6 View commit details
    Browse the repository at this point in the history
  7. Fix key names: getDiskDataSec -> getRawDataBBox and getDiskBiasSec ->…

    … getRawHorizontalOverscanBBox
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    7929f69 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    c038017 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    eb85057 View commit details
    Browse the repository at this point in the history
  10. add defect files to flag another bad amp in CCD 9 starting semester 14B

    CCD 9 has lost another amplifier (now 2 out of 4).  This update
    will allow us to process S14A and S14B data without hacking the
    mapper etc.
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    d826d27 View commit details
    Browse the repository at this point in the history
  11. Cherry-pick to update hsc/defects files

    Move defect definition of CCD 9 == 1_47, amp 3 to new file in
    the directory when that amp got bad (2014-04-03).
    validStart will be set to that date.
    
    Conflicts:
    	bin/genDefectRegistry.py
    	hsc/defects/2013-01-31/defects.dat
    yasudank authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    5cdecae View commit details
    Browse the repository at this point in the history
  12. Cherry-pick to update genDefectRegistry.py

    genDefectRegistry: support multiple versions of same CCD defects
    
    Multiple versions of the same CCD defects were thrown in the
    database with the same end date (way in the future), so that
    the query to find the defect ended up with multiple rows.
    Instead, have one defect valid until the next, and only the
    last is valid into the future.
    
    Conflicts:
    	bin/genDefectRegistry.py
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    7f7db88 View commit details
    Browse the repository at this point in the history
  13. Update FITS header NAME in genDefectFits.py to match cameraMapper.py …

    …expectation
    
    The function bypass_defects wants the FITS header "name" to be equivalent to the
    detectorName.  Also removed creation of the superfluous header "SERIAL".
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    a047ae9 View commit details
    Browse the repository at this point in the history
  14. defects: fix dependencies in build

    Only the last defects FITS file was being built.  Naming them distinctly,
    and making them all dependencies of the registry ensures they are all
    built.
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    8dced42 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    6ca559a View commit details
    Browse the repository at this point in the history
  16. Add functionality to obtain amplifier linearity parameters

    Amplifier linearity parameters are now read in from the
    hsc_geom.paf policy file and stored in the curretnly
    available AmpInfoTable fields: linearityType and
    linearityCoeffs.  These may be changed when the linearity
    corrections are defined and implemented in LSST.
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    9ea545b View commit details
    Browse the repository at this point in the history
  17. Fix ccd name keys in specialChipMap: hscNNN -> NNN

    Also changes the variable the key is being compared to to be
    detConfig.serial which is what NNN is referring to (rather
    than detConfig.name).
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    c85e977 View commit details
    Browse the repository at this point in the history
  18. genCameraRepository: import shutil

    shutil is required for the --clobber command line option.
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    87331d7 View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    35cb80b View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    9815110 View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    c75a827 View commit details
    Browse the repository at this point in the history
  22. isr: Update linearize function

    Now reads in Linearity Parameters from AmpInfoTable
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    b69414e View commit details
    Browse the repository at this point in the history
  23. Use ip_isr's maskAndInterpDefect function

    Delete local maskDefect function in favor of using the (essentially
    identical) maskAndInterpDefect function in ip_isr's isrTask.py.
    Use doDefect config clause when calling maskAndInterpDefect.
    Set doDefect config default to True.
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    8a8af12 View commit details
    Browse the repository at this point in the history
  24. HscMapper: add fringe_config product

    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    b2965d5 View commit details
    Browse the repository at this point in the history
  25. config: add fringe for HSC

    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    eb623ba View commit details
    Browse the repository at this point in the history
  26. config: activate bias, dark for HSC

    Conflicts:
    	config/hsc/isr.py
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    6d19f1a View commit details
    Browse the repository at this point in the history
  27. config: remove HSC bad amp

    It's better now.
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    709d3ba View commit details
    Browse the repository at this point in the history
  28. Configuration menu
    Copy the full SHA
    5d70580 View commit details
    Browse the repository at this point in the history
  29. Configuration menu
    Copy the full SHA
    ff74cc5 View commit details
    Browse the repository at this point in the history
  30. config: ignore INTRP pixels when coadding

    e.g., CRs
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    94a010c View commit details
    Browse the repository at this point in the history
  31. Configuration menu
    Copy the full SHA
    e08fe9d View commit details
    Browse the repository at this point in the history
  32. Configuration menu
    Copy the full SHA
    079e307 View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    3793203 View commit details
    Browse the repository at this point in the history
  34. Configuration menu
    Copy the full SHA
    5949953 View commit details
    Browse the repository at this point in the history
  35. Configuration menu
    Copy the full SHA
    3ab0be7 View commit details
    Browse the repository at this point in the history
  36. Configuration menu
    Copy the full SHA
    130be1d View commit details
    Browse the repository at this point in the history
  37. config: Specify 'NO_DATA' for badMaskPlanes of fringe subtraction

    Conflicts:
    	config/hsc/processCcd.py
    yasudank authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    f7f3b18 View commit details
    Browse the repository at this point in the history
  38. Configuration menu
    Copy the full SHA
    0b827ea View commit details
    Browse the repository at this point in the history
  39. isr: fix masking for saturation trail widening

    Was clobbering mask bits instead of OR-ing.  This
    affected background subtraction in areas that were
    masked (e.g., BAD).
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    ff36f47 View commit details
    Browse the repository at this point in the history
  40. Configuration menu
    Copy the full SHA
    2f2ef37 View commit details
    Browse the repository at this point in the history
  41. Adjusted saturation levels for some CCDs based on outliers in (psf-ap…

    …) followed by looking at raw frames; DM-333
    RobertLuptonTheGood authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    086e475 View commit details
    Browse the repository at this point in the history
  42. Configuration menu
    Copy the full SHA
    2bb4289 View commit details
    Browse the repository at this point in the history
  43. isr: better defect interpolation

    Defects that had been interpolated were being replaced by the image mean.
    
    Do defect interpolation after flat-fielding, so they don't get tweaked
    by the flat-field.
    
    Conflicts:
    	python/lsst/obs/subaru/isr.py
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    ecb855a View commit details
    Browse the repository at this point in the history
  44. Configuration menu
    Copy the full SHA
    0acfef7 View commit details
    Browse the repository at this point in the history
  45. added eups_versions mapper target

    Fixup: remove whitespace
    StevenBickerton authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    f4a0fc0 View commit details
    Browse the repository at this point in the history
  46. Adjusted saturation levels for some CCDs based on outliers in (psf-ap…

    …) followed by looking at raw frames; DM-333
    RobertLuptonTheGood authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    b70099f View commit details
    Browse the repository at this point in the history
  47. Configuration menu
    Copy the full SHA
    517f06f View commit details
    Browse the repository at this point in the history
  48. List seeing in arcsec, not fwhm

    Also remove frameid from default list of fields (you
    can add it back with -a fieldid)
    RobertLuptonTheGood authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    4039c07 View commit details
    Browse the repository at this point in the history
  49. Configuration menu
    Copy the full SHA
    3eb4363 View commit details
    Browse the repository at this point in the history
  50. Configuration menu
    Copy the full SHA
    2c5089e View commit details
    Browse the repository at this point in the history
  51. Specify valid registry DB columns for meas_mosaic outputs, so we don'…

    …t try to query the registry by tract
    
    Conflicts:
    	policy/HscMapper.paf
    TallJimbo authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    5fc8e08 View commit details
    Browse the repository at this point in the history
  52. Configuration menu
    Copy the full SHA
    46cdca4 View commit details
    Browse the repository at this point in the history
  53. Configuration menu
    Copy the full SHA
    2da7a52 View commit details
    Browse the repository at this point in the history
  54. Configuration menu
    Copy the full SHA
    a02497f View commit details
    Browse the repository at this point in the history
  55. Disable background matching in coaddition by default.

    Conflicts:
    	config/assembleCoadd.py
    TallJimbo authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    bef5cd3 View commit details
    Browse the repository at this point in the history
  56. Configuration menu
    Copy the full SHA
    c2c8b6a View commit details
    Browse the repository at this point in the history
  57. mappers: remove stack products

    These are left over from hscMosaic, which is no longer used.
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    4d3bbd9 View commit details
    Browse the repository at this point in the history
  58. Configuration menu
    Copy the full SHA
    5633cd8 View commit details
    Browse the repository at this point in the history
  59. Configuration menu
    Copy the full SHA
    402d563 View commit details
    Browse the repository at this point in the history
  60. Change starSelector['objectSize'].widthMin to smaller value to avoid …

    …a failure for good seeing data.
    yasudank authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    34d46bf View commit details
    Browse the repository at this point in the history
  61. HscMapper: set midtime on raw exposures

    The superclass CameraMapper attempts to set the midtime from MJD-OBS,
    but HSC uses MJD-STR.
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    2eca197 View commit details
    Browse the repository at this point in the history
  62. Change starSelector['objectSize'].fluxMin to smaller value to use fai…

    …nter stars especially for g-band.
    yasudank authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    1ea00bf View commit details
    Browse the repository at this point in the history
  63. Configuration menu
    Copy the full SHA
    c481591 View commit details
    Browse the repository at this point in the history
  64. isr: calculate overscan median with bad pixels removed

    The bad pixels (e.g., saturation) will otherwise bias the
    median; this can be severe when there's a bright source
    close to the edge of the amp.
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    85f4520 View commit details
    Browse the repository at this point in the history
  65. Update fitsthumb call to use new API.

    A recent update in fitsthumb changed the API.  The format is no longer
    provided and I've removed it completely as the butler uses png.  To
    do otherwise would mean the '_filename' returned by a butler.get()
    would be incorrect.  Also, createFitsThumb() now takes an ndarray.
    StevenBickerton authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    82ab5fd View commit details
    Browse the repository at this point in the history
  66. crosstalk: don't use use saturated amps

    Amps that are completely saturated do not provide any useful
    information for the crosstalk correction, and using them taints
    the good amps (NANs or mask).  This shouldn't affect the result,
    as the image is completely saturated and there's not much information
    there to use in the correction.
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    01b970a View commit details
    Browse the repository at this point in the history
  67. Configuration menu
    Copy the full SHA
    0acf351 View commit details
    Browse the repository at this point in the history
  68. add datasets to support multi-band processing

    In addition to new source catalogs, also includes config, metadata and
    schema entries for the new CmdLineTasks.
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    be1bc32 View commit details
    Browse the repository at this point in the history
  69. Configuration menu
    Copy the full SHA
    5fa5c84 View commit details
    Browse the repository at this point in the history
  70. Configuration menu
    Copy the full SHA
    8c19dce View commit details
    Browse the repository at this point in the history
  71. Configuration menu
    Copy the full SHA
    57a8c82 View commit details
    Browse the repository at this point in the history
  72. Configuration menu
    Copy the full SHA
    14a4a3d View commit details
    Browse the repository at this point in the history
  73. Configuration menu
    Copy the full SHA
    4c117bf View commit details
    Browse the repository at this point in the history
  74. HscMapper: add validity range for bias, dark

    Couldn't have more than one bias or dark in the registry
    ("RintimeError: No unique lookup for bias from ['calibDate', 'calibVersion'] from {'filter': 'NONE', 'ccd': 108, 'visit': 1136}: 2 matches").
    Adding validty ranges fixes this.
    PaulPrice authored and laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    0f30ce2 View commit details
    Browse the repository at this point in the history
  75. isr: Remove reference to deprecated function display()

    Now calls lsst.afw.display.ds9 directly.
    laurenam committed Feb 4, 2015
    Configuration menu
    Copy the full SHA
    24140b5 View commit details
    Browse the repository at this point in the history

Commits on Feb 18, 2015

  1. Account for rotated calib images for bias/dark/flat correction

    Approximately half of the HSC CCDs are rotated 180 deg with respect
    to the others.  Two others have 90 deg rotations and another two have
    270 deg rotations .  Their associated calibration frames thus need
    to be rotated prior to applying the corrections.
    laurenam committed Feb 18, 2015
    Configuration menu
    Copy the full SHA
    746852a View commit details
    Browse the repository at this point in the history