Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-40739: Add column forwarding in exposure summary stats to allow decl to be renamed to dec on read. #708

Merged
merged 2 commits into from
Sep 12, 2023
Merged

DM-40739: Add column forwarding in exposure summary stats to allow decl to be renamed to dec on read. #708

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5aa57cb
Add ability to forward fields while reading exposure summary stats.
erykoff Sep 11, 2023
9c9b01e
Add tests for exposure summary stats column forwarding.
erykoff Sep 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 8 additions & 1 deletion python/lsst/afw/image/_exposureSummaryStats.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,12 +145,19 @@ def _write(self):
@staticmethod
def _read(bytes):
yamlDict = yaml.load(bytes, Loader=yaml.SafeLoader)

# Special list of fields to forward to new names.
forwardFieldDict = {"decl": "dec"}

# For forwards compatibility, filter out any fields that are
# not defined in the dataclass.
droppedFields = []
for _field in list(yamlDict.keys()):
if _field not in ExposureSummaryStats.__dataclass_fields__:
droppedFields.append(_field)
if _field in forwardFieldDict and forwardFieldDict[_field] not in yamlDict:
yamlDict[forwardFieldDict[_field]] = yamlDict[_field]
else:
droppedFields.append(_field)
yamlDict.pop(_field)
if len(droppedFields) > 0:
droppedFieldString = ", ".join([str(f) for f in droppedFields])
Expand Down
26 changes: 26 additions & 0 deletions tests/test_exposure.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import dataclasses
import os.path
import unittest
import warnings

import numpy as np
from numpy.testing import assert_allclose
Expand Down Expand Up @@ -1158,6 +1159,31 @@ def testExposureSummaryExtraComponents(self):
self.assertEqual(summaryStats.ra, testDict['ra'])
self.assertEqual(summaryStats.dec, testDict['dec'])

def testExposureSummaryForwardComponents(self):
"""Test that we can forward extra components (e.g. decl->dec).
"""
testDict = {'ra': 10.0,
'decl': 10.0}
bytes = yaml.dump(testDict, encoding='utf-8')
# Cleanly forwarded fields must not result in a warning.
with warnings.catch_warnings():
warnings.simplefilter("error")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a comment here that forwarding fields must not result in a warning.

summaryStats = lsst.afw.image.ExposureSummaryStats._read(bytes)

self.assertEqual(summaryStats.ra, testDict['ra'])
self.assertEqual(summaryStats.dec, testDict['decl'])

# And check if there are both listed, it should use the new dec value.
testDict = {'ra': 10.0,
'dec': 5.0,
'decl': 10.0}
bytes = yaml.dump(testDict, encoding='utf-8')
with self.assertWarns(FutureWarning):
summaryStats = lsst.afw.image.ExposureSummaryStats._read(bytes)

self.assertEqual(summaryStats.ra, testDict['ra'])
self.assertEqual(summaryStats.dec, testDict['dec'])

def testExposureSummarySchema(self):
"""Test that we can make a schema for an exposure summary and populate
records with that schema.
Expand Down