Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-35278: Increase version of ptcDataset and make it backwards compatible #217

Merged
merged 2 commits into from
Jun 22, 2022
Merged

DM-35278: Increase version of ptcDataset and make it backwards compatible #217

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
307903b
Make PTC dataset backward compatible
plazas Jun 21, 2022
7a23727
Add check and docstring with version
plazas Jun 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 14 additions & 2 deletions python/lsst/ip/isr/ptcDataset.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,11 +150,15 @@ class PhotonTransferCurveDataset(IsrCalib):
-------
`lsst.cp.pipe.ptc.PhotonTransferCurveDataset`
Output dataset from MeasurePhotonTransferCurveTask.

Notes
-----
Version 1.1 adds the `ptcTurnoff` attribute.
"""

_OBSTYPE = 'PTC'
_SCHEMA = 'Gen3 Photon Transfer Curve'
_VERSION = 1.0
_VERSION = 1.1

def __init__(self, ampNames=[], ptcFitType=None, covMatrixSide=1, **kwargs):

Expand Down Expand Up @@ -440,6 +444,10 @@ def fromTable(cls, tableList):
inDict['badAmps'] = []
inDict['photoCharge'] = dict()

calibVersion = metadata['PTC_VERSION']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need a check on calibVersion for this warning message? It seems like that message should only trigger if calibVersion == 1.0.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good catch, thanks. I had the message inside the loop, with the check, but when I decided to just have one message outside the loop, I did not put a check.

if calibVersion == 1.0:
cls().log.warning(f"Previous version found for PTC dataset: {calibVersion}. "
f"Setting 'ptcTurnoff' in all amps to last value in 'finalMeans'.")
for record in ptcTable:
ampName = record['AMPLIFIER_NAME']

Expand All @@ -458,7 +466,6 @@ def fromTable(cls, tableList):
inDict['ptcFitPars'][ampName] = record['PTC_FIT_PARS']
inDict['ptcFitParsError'][ampName] = record['PTC_FIT_PARS_ERROR']
inDict['ptcFitChiSq'][ampName] = record['PTC_FIT_CHI_SQ']
inDict['ptcTurnoff'][ampName] = record['PTC_TURNOFF']
inDict['covariances'][ampName] = record['COVARIANCES']
inDict['covariancesModel'][ampName] = record['COVARIANCES_MODEL']
inDict['covariancesSqrtWeights'][ampName] = record['COVARIANCES_SQRT_WEIGHTS']
Expand All @@ -471,6 +478,11 @@ def fromTable(cls, tableList):
inDict['finalMeans'][ampName] = record['FINAL_MEANS']
inDict['badAmps'] = record['BAD_AMPS']
inDict['photoCharge'][ampName] = record['PHOTO_CHARGE']
if calibVersion == 1.0:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a docstring to the class that mentions that v1.1 adds the ptcTurnoff attribute? I think it'll be easier to understand changes if it doesn't require looking for code tests on calibVersion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. On a more general note, are the docstrings the only places where version changes are tracked? I just want to make sure that we don't have to update another document (or perhaps this is the time to decide if we want o create + maintain such an external document).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know of a better place, so I think adding the changes to the docstring is the minimum we should be doing. I don't think we'll have so many of these that we'll need to have major release notes.

mask = record['FINAL_MEANS'].mask
inDict['ptcTurnoff'][ampName] = record['FINAL_MEANS'][~mask][-1]
else:
inDict['ptcTurnoff'][ampName] = record['PTC_TURNOFF']
return cls().fromDict(inDict)

def toTable(self):
Expand Down