Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-18638: Check for at least 2 visits in AMx calc #103

Merged
merged 1 commit into from Mar 29, 2019

Conversation

yalsayyad
Copy link
Contributor

before including a pair's distance stdev. The distance of a pair that
appears in only one visit has a std of zero, biasing the median RMS
if used.

before including a pair's distance stdev. The distance of a pair that
appears in only one visit has a std of zero, biasing the median RMS
if used.
@yalsayyad yalsayyad changed the title Check for at least 2 visits in AMx calc DM-18638: Check for at least 2 visits in AMx calc Mar 29, 2019
@yalsayyad yalsayyad merged commit 015ca72 into master Mar 29, 2019
@RobertLuptonTheGood
Copy link

The reason for using the unbiased stdev 1/(n-1) \sum(x - \bar x)^2 is partially to avoid this problem. Are you using 1/n?

@yalsayyad
Copy link
Contributor Author

We need at least 2 values regardless of what estimator we use.

Looking at the line below, it uses np.std, which I know you've complained uses 1/n by default:
https://docs.scipy.org/doc/numpy/reference/generated/numpy.std.html

@wmwv
Copy link
Contributor

wmwv commented Apr 1, 2019

Yes, this was my fault. I thought that numpy.std used 1/(n-1) and would return NaN for n=1. But I did not test this implicit assumption.

I suggest we require 2 values and then return n/(n-1) * np.std(...) to correct the bias.

@yalsayyad
Copy link
Contributor Author

@wmwv Do want to file a ticket and push that change or shall I?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants