Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-34863: Update tests and remove gen2 code from obs_decam. #227

Merged
merged 34 commits into from
Jul 20, 2022
Merged

DM-34863: Update tests and remove gen2 code from obs_decam. #227

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
023f9d7
Update eups table for new tests.
erykoff Jul 14, 2022
85d547c
Add gen3 tests for basic isr + crosstalk.
erykoff Jul 18, 2022
4154dc9
Remove old gen2 tests of isr and crosstalk.
erykoff Jul 18, 2022
f848043
Replace test to check for linearizers.
erykoff Jul 18, 2022
8c6fb3e
Remove gen2 to 3 conversion tests.
erykoff Jul 18, 2022
dd44139
Remove gen2 getId test.
erykoff Jul 18, 2022
cd55312
Update test for gen3.
erykoff Jul 18, 2022
bccee66
Rename test file
erykoff Jul 18, 2022
a4ae5ed
Remove gen2 test for getting instCal.
erykoff Jul 18, 2022
7aa7cd5
Remove gen2 calib test.
erykoff Jul 18, 2022
0ad40b5
Update wcs card test for gen3.
erykoff Jul 18, 2022
17d8045
Fix skipping logic when testdata_decam isn't setup.
erykoff Jul 18, 2022
ea47144
Update test SConscript.
erykoff Jul 18, 2022
a77c9ea
Add missing import for assertDetectorsEqual mixin.
erykoff Jul 18, 2022
0545354
Remove gen2 null isr task.
erykoff Jul 18, 2022
3c493de
Remove gen2 mapper.
erykoff Jul 18, 2022
1230516
Remove gen2 ingestCalibs.
erykoff Jul 18, 2022
bd8f7c4
Remove gen2 code and deprecate DecamCrosstalkTask.
erykoff Jul 18, 2022
9276bb8
Remove deleted import.
erykoff Jul 18, 2022
9550551
Remove gen2 ingest task.
erykoff Jul 18, 2022
54768ab
Update makeLinearizer.py to be gen3-only.
erykoff Jul 18, 2022
af4b3fa
Fix bug in makeLinearizer force.
erykoff Jul 18, 2022
c62ef14
Update makeCrosstalkDecam for gen3 usage.
erykoff Jul 18, 2022
afa955f
Remove gen2 ingestion.
erykoff Jul 18, 2022
501ef7c
Update gitignore to ignore linearity files.
erykoff Jul 18, 2022
bca78d5
Remove gen2-only executables.
erykoff Jul 18, 2022
fb22646
Remove unnecessary SConscript.
erykoff Jul 18, 2022
3b22e5b
Add instructions to README.
erykoff Jul 18, 2022
2fb557a
Remove gen2 decam mapper policy.
erykoff Jul 18, 2022
911d317
Removed unused dependencies from tests.
erykoff Jul 18, 2022
5120f99
Change installation to skip decam/ directory.
erykoff Jul 18, 2022
ddb8844
Update documentation to remove gen2 tasks.
erykoff Jul 18, 2022
06c9f5b
Use SConstruct to limit installation instead of eupspkg.
erykoff Jul 18, 2022
10a47ae
Remove bin from SConstruct subdirs.
erykoff Jul 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,5 @@ decam/CALIB/defects
decam/CALIB/crosstalk
decam/CALIB/calibRegistry*
decam/crosstalk
decam/linearity
.vscode/
2 changes: 1 addition & 1 deletion SConstruct
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# -*- python -*-
from lsst.sconsUtils import scripts
scripts.BasicSConstruct("obs_decam", defaultTargets=scripts.DEFAULT_TARGETS + ("decam",),
disableCc=True)
disableCc=True, subDirList=['config', 'doc', 'pipelines', 'python', 'ups'])
3 changes: 0 additions & 3 deletions bin.src/ingestImagesDecam.py
View file
Open in desktop

This file was deleted.

5 changes: 0 additions & 5 deletions bin.src/runCrosstalkAlone.py
View file
Open in desktop

This file was deleted.

2 changes: 0 additions & 2 deletions config/isr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
"""
import os.path

from lsst.obs.decam.crosstalk import DecamCrosstalkTask

config.datasetType = "raw"
config.fallbackFilterName = None
Expand Down Expand Up @@ -60,7 +59,6 @@

config.doCrosstalkBeforeAssemble = True
config.doCrosstalk = True
config.crosstalk.retarget(DecamCrosstalkTask)
config.crosstalk.minPixelToMask=45000.0
config.crosstalk.crosstalkMaskPlane="CROSSTALK"

Expand Down
1 change: 0 additions & 1 deletion decam/CALIB/_mapper
View file
Open in desktop

This file was deleted.

3 changes: 3 additions & 0 deletions decam/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@ The linearity table `linearity_table_v*.fits` is a standard DECam calibration fi
[DECam Community Pipeline Calibration Files](http://www.ctio.noao.edu/noao/content/decam-calibration-files).
Building the `obs_decam` package converts the linearity table to LSST linearizers `linearizer/*.fits`
by running `makeLinearizer.py`.

To convert inputs to the format suitable for `obs_decam_data`, run `makeLinearizer.py linearity_table_v0.4.fits` and `makeCrosstalkDecam.py DECam_xtalk_20130606.txt`.

72 changes: 0 additions & 72 deletions decam/SConscript
View file
Open in desktop

This file was deleted.

22 changes: 17 additions & 5 deletions decam/makeCrosstalkDecam.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,15 @@
import os.path
import sys

import lsst.obs.decam
import lsst.ip.isr as ipIsr
from lsst.daf.base import PropertyList
from lsst.obs.decam import DecamMapper
from lsst.utils import getPackageDir


def getCrosstalkDir():
"""Get crosstalk directory in obs package for outputting data."""
return os.path.join(getPackageDir('obs_decam'), 'decam', 'crosstalk')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's highly likely that you are going to want people to specify an input directory or a glob expanding to all the files rather than expecting the crosstalk files to always be here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the destination, it just puts them in decam/crosstalk as before.



def makeDetectorCrosstalk(dataDict, force=False):
Expand All @@ -26,7 +32,7 @@ def makeDetectorCrosstalk(dataDict, force=False):
decamCT.updateMetadata(setDate=False, CALIBDATE='1970-01-01T00:00:00')

detName = dataDict['DETECTOR_NAME']
outDir = os.path.join(DecamMapper.getCrosstalkDir(), detName.lower())
outDir = os.path.join(getCrosstalkDir(), detName.lower())
if os.path.exists(outDir):
if not force:
print("Output directory %r exists; use --force to replace" % (outDir, ))
Expand Down Expand Up @@ -55,10 +61,16 @@ def readFile(crosstalkInfile):
RuntimeError :
Raised if the detector is not known.
"""
instrument = lsst.obs.decam.DarkEnergyCamera()
camera = instrument.getCamera()

ampIndexMap = {'A': 0, 'B': 1}
detMap = {f"ccd{key:02d}": value for key, value in DecamMapper.detectorNames.items()}
detMap = {}
detSerialMap = {}
for detector in camera:
detMap[f"ccd{detector.getId():02d}"] = detector.getName()
detSerialMap[detector.getName()] = detector.getId()
detMap['ccd61'] = 'N30'
detSerialMap = {value: key for key, value in DecamMapper.detectorNames.items()}
detSerialMap['N30'] = 61

outDict = dict()
Expand Down Expand Up @@ -131,7 +143,7 @@ def readFile(crosstalkInfile):

outDict = readFile(crosstalkInfile=cmd.crosstalkInfile)

crosstalkDir = DecamMapper.getCrosstalkDir()
crosstalkDir = getCrosstalkDir()
if os.path.exists(crosstalkDir):
if not cmd.force:
print("Output directory %r exists; use --force to replace" % (crosstalkDir, ))
Expand Down