Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

obs_subaru/DM-7884: unification of coadd and diff datasets #40

Merged
merged 3 commits into from
Oct 30, 2016
Merged

obs_subaru/DM-7884: unification of coadd and diff datasets #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
15af83a
Remove datasets which appear to be obsolete
pgee2000 Oct 19, 2016
69edb5c
Remove items from datasets which do not differ from obs_base
pgee2000 Oct 20, 2016
3933111
Review comments, include from KT about tables, raw_visit and raw_skyTile
pgee2000 Oct 29, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
77 changes: 3 additions & 74 deletions policy/HscMapper.paf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,37 +143,9 @@ exposures: {
columns: "pointing" "filter" "visit" "ccd"
}

deepCoadd_depth: {
template: "deepCoadd/%(filter)s/%(tract)d/%(patch)s.depth.fits"
python: "lsst.afw.image.ImageF"
persistable: "ImageF"
storage: "FitsStorage"
level: "Skytile"
tables: raw_skytile
}
# dataset defined in obs_base modified in this mapper as follows:
deepCoadd_tempExp: {
template: "deepCoadd/%(filter)s/%(tract)d/%(patch)s/warp-%(filter)s-%(tract)d-%(patch)s-%(visit)d.fits"
python: "lsst.afw.image.ExposureF"
persistable: "ExposureF"
storage: "FitsStorage"
level: "Skytile"
tables: raw_skytile
}
deepCoadd_bgRef: {
template: "deepCoadd/%(filter)s/%(tract)d/%(patch)s/bgRef-%(tract)d-%(patch)s-%(filter)s.fits"
python: "lsst.afw.image.ExposureF"
persistable: "ExposureF"
storage: "FitsStorage"
level: "Skytile"
tables: raw_skytile
}
deepCoadd_bg: {
template: "deepCoadd/%(filter)s/%(tract)d/%(patch)s/bgModel-%(tract)d-%(patch)s-%(filter)s.fits"
python: "lsst.afw.image.ImageF"
persistable: "ImageF"
storage: "FitsStorage"
level: "Skytile"
tables: raw_skytile
}

deepCoadd_diff: {
Expand All @@ -193,14 +165,6 @@ exposures: {
tables: raw_skytile
}

diff: {
template: "diff/%(diff)05d/DIFF%(patch)08d.fits"
python: "lsst.afw.image.ExposureF"
persistable: "ExposureF"
storage: "FitsStorage"
level: "Skytile"
tables: "raw_skytile"
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't leave two blank lines.

mosaicCalib: {
template: "%(pointing)05d/%(filter)s/output/CALIB-%(visit)07d-%(ccd)03d-%(tract)d.fits"
Expand Down Expand Up @@ -450,28 +414,7 @@ datasets: {
tables: "raw_skytile"
}

diffsources: {
template: "diff/%(diff)05d/SRC%(patch)08d%(filter)s.fits"
python: "lsst.afw.table.SourceCatalog"
persistable: "SourceCatalog"
storage: "FitsCatalogStorage"
tables: "raw_skytile"
}
diffsources_schema: {
template: "schema/diffsources.fits"
python: "lsst.afw.table.SourceCatalog"
persistable: "SourceCatalog"
storage: "FitsCatalogStorage"
tables: "raw_skytile"
}

diffpsf: {
template: "diff/%(diff)05d/PSF%(patch)08d%(filter)s.boost"
python: "lsst.afw.detection.Psf"
persistable: "Psf"
storage: "BoostStorage"
tables: "raw_skytile"
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

warppsf: {
template: "%(pointing)05d/%(filter)s/warp/WARPPSF-%(visit)07d-%(ccd)03d.boost"
Expand Down Expand Up @@ -551,27 +494,13 @@ datasets: {



# dataset defined in obs_base modified in this mapper as follows:
deepCoadd_srcMatch: {
template: "deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/srcMatch-%(filter)s-%(tract)d-%(patch)s.fits"
python: "lsst.afw.table.BaseCatalog"
persistable: "BaseCatalog"
storage: "FitsCatalogStorage"
tables: "raw"
tables: "raw_visit"
}
deepCoadd_icMatch: {
template: "deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/icMatch-%(filter)s-%(tract)d-%(patch)s.fits"
python: "lsst.afw.table.BaseCatalog"
persistable: "BaseCatalog"
storage: "FitsCatalogStorage"
tables: "raw"
tables: "raw_visit"
}
# dataset defined in obs_base modified in this mapper as follows:
deepCoadd_calexp_background: {
template: "deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/det_bkgd-%(filter)s-%(tract)d-%(patch)s.fits"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the HSC template different? Is there a reason not to just adopt the obs_base one?

python: "lsst.afw.math.BackgroundList"
persistable: "PurePythonClass"
storage: "FitsCatalogStorage"
}


Expand Down
85 changes: 5 additions & 80 deletions policy/SuprimecamMapper.paf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,39 +138,11 @@ exposures: {
columns: "pointing" "filter" "visit" "ccd"
}

deepCoadd_depth: {
template: "deepCoadd/%(filter)s/%(tract)d/%(patch)s.depth.fits"
python: "lsst.afw.image.ImageF"
persistable: "ImageF"
storage: "FitsStorage"
level: "Skytile"
tables: raw_skytile
}
# dataset defined in obs_base modified in this mapper as follows:
deepCoadd_tempExp: {
template: "deepCoadd/%(filter)s/%(tract)d/%(patch)s/warp-%(filter)s-%(tract)d-%(patch)s-%(visit)d.fits"
python: "lsst.afw.image.ExposureF"
persistable: "ExposureF"
storage: "FitsStorage"
level: "Skytile"
tables: raw_skytile
}
deepCoadd_bgRef: {
template: "deepCoadd/%(filter)s/%(tract)d/%(patch)s/bgRef-%(tract)d-%(patch)s-%(filter)s.fits"
python: "lsst.afw.image.ExposureF"
persistable: "ExposureF"
storage: "FitsStorage"
level: "Skytile"
tables: raw_skytile
}

diff: {
template: "diff/%(diff)05d/DIFF%(patch)08d.fits"
python: "lsst.afw.image.ExposureF"
persistable: "ExposureF"
storage: "FitsStorage"
level: "Skytile"
tables: "raw_skytile"
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

mosaicCalib: {
template: "%(pointing)05d/%(filter)s/output/CALIB%(visit)07d%(ccd)1d-%(tract)d.fits"
Expand Down Expand Up @@ -402,28 +374,7 @@ datasets: {
storage: "None"
}

diffsources: {
template: "diff/%(diff)05d/SRC%(patch)08d%(filter)s.fits"
python: "lsst.afw.table.SourceCatalog"
persistable: "SourceCatalog"
storage: "FitsCatalogStorage"
tables: "raw_skytile"
}
diffsources_schema: {
template: "schema/diffsources.fits"
python: "lsst.afw.table.SourceCatalog"
persistable: "SourceCatalog"
storage: "FitsCatalogStorage"
tables: "raw_skytile"
}

diffpsf: {
template: "diff/%(diff)05d/PSF%(patch)08d%(filter)s.boost"
python: "lsst.afw.detection.Psf"
persistable: "Psf"
storage: "BoostStorage"
tables: "raw_skytile"
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

warppsf: {
template: "%(pointing)05d/%(filter)s/warp/WARPPSF-%(visit)07d-%(ccd)03d.boost"
Expand Down Expand Up @@ -498,29 +449,9 @@ datasets: {
}


deepCoadd_icMatch: {
template: "deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/icMatch-%(filter)s-%(tract)d-%(patch)s.fits"
python: "lsst.afw.table.SourceCatalog"
persistable: "ignored"
storage: "FitsCatalogStorage"
tables: raw
tables: raw_skyTile
}
# dataset defined in obs_base modified in this mapper as follows:
deepCoadd_srcMatch: {
template: "deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/srcMatch-%(filter)s-%(tract)d-%(patch)s.fits"
python: "lsst.afw.table.BaseCatalog"
persistable: "BaseCatalog"
storage: "FitsCatalogStorage"
tables: "raw"
tables: "raw_visit"
}
deepCoadd_icMatch: {
template: "deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/icMatch-%(filter)s-%(tract)d-%(patch)s.fits"
python: "lsst.afw.table.BaseCatalog"
persistable: "BaseCatalog"
storage: "FitsCatalogStorage"
tables: "raw"
tables: "raw_visit"
}
deepCoadd_calexpBackground: {
template: "deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/bkgd-%(filter)s-%(tract)d-%(patch)s.fits"
Expand Down Expand Up @@ -572,17 +503,13 @@ datasets: {
tables: raw
tables: raw_visit
}
# dataset defined in obs_base modified in this mapper as follows:
forced_src_schema: {
template: "schema/forced_src.fits"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I think we should use what's in obs_base unless there's a good reason not to.

python: "lsst.afw.table.SourceCatalog"
persistable: "ignored"
storage: "FitsCatalogStorage"
}
# dataset defined in obs_base modified in this mapper as follows:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This python entry is incorrect. It should point to a class in meas.base, not pipe.tasks. Hopefully that change will bring it into agreement with obs_base.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have this fix listed on RFC-237. Will do it next week.

forcedPhotCcd_config: {
template: "config/forcedPhotCcd.py"
python: "lsst.pipe.tasks.forcedPhotCcd.ForcedPhotCcdConfig"
persistable: "Config"
storage: "ConfigStorage"
}
forcedPhotCcd_metadata: {
template: "%(pointing)05d/%(filter)s/tract%(tract)d/forcedPhotCcd_metadata/%(visit)07d%(ccd)1d.boost"
Expand Down Expand Up @@ -750,11 +677,9 @@ datasets: {
# Measurements on coadds

# Reference catalog for coadds
# dataset defined in obs_base modified in this mapper as follows:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This python entry is definitely incorrect: it refers to the base class of the config class we want. I suspect obs_base has the right one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have this fix listed on RFC-237. Will do it next week.

mergeCoaddMeasurements_config: {
template: "config/mergeMeasure.py"
python: "lsst.pipe.tasks.multiBand.MergeSourcesConfig"
persistable: "Config"
storage: "ConfigStorage"
}

multiband_config: {
Expand Down