Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-27783: Add improvements to getInitialGoodPoints in ptc.py #67

Merged
merged 5 commits into from Dec 3, 2020

Conversation

plazas
Copy link
Contributor

@plazas plazas commented Dec 1, 2020

No description provided.

@plazas plazas requested a review from czwa December 1, 2020 20:48
Copy link
Contributor

@czwa czwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor issues and questions.

Minimum signal value (in ADU) after which to start examining
the ratios var/mean.

minVarPivotSearch : `float`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See below.

if len(pivot) > 0:
# For small values, sometimes the variance decreases slightly
# Only look when var > self.config.minVarPivotSearch
pivot = [p for p in pivot if variances[p] > minVarPivotSearch]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just confirming that applying the minimum in the variance space is correct. Cutting on the mean seems better to me, as this is a saturation effect, and we want to not cut for saturation before we think we're at that point. This also feels odd because we're looking for a turnover but only if the turnover is above that minimum value. I'm fine either way, so feel free to ignore this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"This also feels odd because we're looking for a turnover but only if the turnover is above that minimum value"---> Right, this is what this minVarPivotSearch cut is meant to help with.
In addition, at this point, the mean vector has been sorted, and the variance vector too using the indices of the mean vector. The code at this point assumes that the variance, after being sorted by the index of the sorted mean, increases monotonically.


ptcFitType : `str`
Fit a 'POLYNOMIAL' (degree: 'polynomialFitDegree') or
'EXPAPPROXIMATION' (Eq. 16 of Astier+19) to the PTC.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this not happen with FULLCOVARIANCE?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This helper function is used by fitPtc, which is used only by POLYNOMIAL or EXPAPPROXIMATION. It was written just to facilitate the reading of fitPtc.

@@ -314,19 +314,19 @@ def test_meanVarMeasurementAllNan(self):
def test_getInitialGoodPoints(self):
xs = [1, 2, 3, 4, 5, 6]
ys = [2*x for x in xs]
points = self.defaultTask._getInitialGoodPoints(xs, ys, 0.1, 0.25)
points = self.defaultTask._getInitialGoodPoints(xs, ys, 0.1, 0.25, 0., 0.)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can these explicitly set the control parameters? _getInitialGoodPoints(xs, ys, maxDeviationPositive=0.2, maxDeviationNegative=0.25, minMeanRatioTest=0.0, minVarPivotSearch=0.0)? This is easier to read and understand than a long list of floats.

@plazas plazas merged commit 597c966 into master Dec 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants