Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TensorTests.cs and Tensor.cs StdDev calculates Variance rather than Standard deviations #110381

Closed
xela-trawets opened this issue Dec 4, 2024 · 5 comments · Fixed by #110392
Closed
Labels
area-System.Numerics.Tensors in-pr There is an active PR which will close this issue when it is merged

Comments

@xela-trawets
Copy link

Description

The StdDev routine lacks a square root at the end. The test also lacks the square root.
Easy to fix, rename to Variance and add a new function to calaculate the STdDev by taking the square root.

However... This is normally* -( see what we did there) used for sample sets and not usually populations So it might be best to add and option to define the sample or popukation usage,, but that is a nuance compare to delivering the wrong thing entirely. (difference being adjustment for degrees of freedom which is division by N or (N-1) - because estimating the mean "uses up" some fraction of the variation)

Reproduction Steps

Calculate the StdDev of almost anything
var t = Enumerable.Range(-32, 64).Select(x => 1.0 * x).ToArray();
var tt = new ReadOnlyTensorSpan (t.AsSpan());
var s = Tensor.StdDev(tt);
Console.WriteLine(s);

341.25

Expected behavior

Something like 18.47

Actual behavior

341.25

Regression?

No response

Known Workarounds

Take the square root,

Configuration

dotnet 9

Other information

No response

@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Dec 4, 2024
@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Dec 4, 2024
@vcsjones vcsjones added area-System.Numerics.Tensors and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Dec 4, 2024
Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors
See info in area-owners.md if you want to be subscribed.

@xela-trawets
Copy link
Author

would you look at a PR for this?

@dotnet-policy-service dotnet-policy-service bot added the in-pr There is an active PR which will close this issue when it is merged label Dec 4, 2024
@aloraman
Copy link

aloraman commented Dec 4, 2024

Btw, in SqlServer, for example, there are two different functions for calculating (un)biased standard deviation.
StDev computes unbiased version (with division by N-1, for population sample)
StDevP computes biased version (with division by N, for population, hence P in the name).

@xela-trawets
Copy link
Author

If the mean is calculated then the N-minus-one, (unbiased), method should be the default.
That is the case in this routine.

@lilinus
Copy link
Contributor

lilinus commented Dec 6, 2024

If the mean is calculated then the N-minus-one, (unbiased), method should be the default. That is the case in this routine.

Agreed. I think it should be clarified in the documentation to avoid confusion (whichever version is chosen).

@dotnet-policy-service dotnet-policy-service bot removed the untriaged New issue has not been triaged by the area owner label Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-System.Numerics.Tensors in-pr There is an active PR which will close this issue when it is merged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants