-
Notifications
You must be signed in to change notification settings - Fork 321
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rank Data, Correlation, Covariance, R Squared #3484
Conversation
e5c14ac
to
3b80fb9
Compare
distribution/lib/Standard/Base/0.0.0-dev/src/Data/Statistics/Rank_Method.enso
Outdated
Show resolved
Hide resolved
distribution/lib/Standard/Base/0.0.0-dev/src/Data/Statistics/Rank_Method.enso
Show resolved
Hide resolved
2fe1ae5
to
4a15578
Compare
## Specifies how to handle ranking of equal values. | ||
type Rank_Method | ||
## Use the mean of all ranks for equal values. | ||
type Average | ||
|
||
## Use the lowest of all ranks for equal values. | ||
type Minimum | ||
|
||
## Use the highest of all ranks for equal values. | ||
type Maximum | ||
|
||
## Use same rank value for equal values and next group is the immediate | ||
following ranking number. | ||
type Dense | ||
|
||
## Equal values are assigned the next rank in order that they occur. | ||
type Ordinal |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This explanation is much better, but I'm still thinking if it would be worth to maybe offer some examples? Not sure if here or next to some method using this. But I'm still not sure if I correctly understand how Dense
or Average
work
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me.
Just two questions:
- I did check the code, but did not read deep into formulas for each statistic, I assume they are correct. But if you want me to double check them, I can do that.
- What are the reference values for the results coming from? I guess Excel? I'm wondering if it would make sense to indicate somehow (in a very short way) how the references are computed if it is something that is not completely trivial, just in case someone wanted to double check them later if some issues were to appear.
case MINIMUM -> start + 1; | ||
case MAXIMUM -> index; | ||
case DENSE -> dense; | ||
case AVERAGE -> (start + 1 + index) / 2.0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm just wondering out of curiosity in which situation one may want to use the average ranking?
(and why is it the default one? not saying it shouldn't be, just a honest question, because I haven't seen that)
Added test cases
Create helpers for promoting dataflow errors as panics. Remove set_at where possible.
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
df9297b
to
e3e75d4
Compare
std-bits/base/src/main/java/org/enso/base/statistics/CorrelationStatistics.java
Show resolved
Hide resolved
- Added new `Statistic`s: Covariance, Pearson, Spearman, R Squared - Added `covariance_matrix` function - Added `pearson_correlation` function to compute correlation matrix - Added `rank_data` and Rank_Method type to create rankings of a Vector - Added `spearman_correlation` function to compute Spearman Rank correlation matrix # Important Notes - Added `Panic.throw_wrapped_if_error` and `Panic.handle_wrapped_dataflow_error` to help with errors within a loop. - Removed `Array.set_at` use from `Table.Vector_Builder`
Pull Request Description
Statistic
s: Covariance, Pearson, Spearman, R Squaredcovariance_matrix
functionpearson_correlation
function to compute correlation matrixrank_data
and Rank_Method type to create rankings of a Vectorspearman_correlation
function to compute Spearman Rank correlation matrixImportant Notes
Panic.throw_wrapped_if_error
andPanic.handle_wrapped_dataflow_error
to help with errors within a loop.Array.set_at
use fromTable.Vector_Builder
Checklist
Please include the following checklist in your PR:
Scala,
Java,
and
Rust
style guides.
./run.sh ide dist
and./run.sh ide watch
.