Rank Data, Correlation, Covariance, R Squared #3484
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jdunkerley
force-pushed
the
wip/jd/covariance-182059993
branch
from
e5c14ac
to
3b80fb9
Compare
radeusgd
reviewed
May 30, 2022
distribution/lib/Standard/Base/0.0.0-dev/src/Data/Statistics/Rank_Method.enso
Outdated
Show resolved
Hide resolved
distribution/lib/Standard/Base/0.0.0-dev/src/Data/Statistics/Rank_Method.enso
Show resolved
Hide resolved
jdunkerley
force-pushed
the
wip/jd/covariance-182059993
branch
from
2fe1ae5
to
4a15578
Compare
radeusgd
reviewed
May 30, 2022
Comment on lines
+2
to
+18
| ## Specifies how to handle ranking of equal values. | ||
| type Rank_Method | ||
| ## Use the mean of all ranks for equal values. | ||
| type Average | ||
|
|
||
| ## Use the lowest of all ranks for equal values. | ||
| type Minimum | ||
|
|
||
| ## Use the highest of all ranks for equal values. | ||
| type Maximum | ||
|
|
||
| ## Use same rank value for equal values and next group is the immediate | ||
| following ranking number. | ||
| type Dense | ||
|
|
||
| ## Equal values are assigned the next rank in order that they occur. | ||
| type Ordinal |
There was a problem hiding this comment.
This explanation is much better, but I'm still thinking if it would be worth to maybe offer some examples? Not sure if here or next to some method using this. But I'm still not sure if I correctly understand how Dense or Average work
radeusgd
approved these changes
May 30, 2022
There was a problem hiding this comment.
Looks good to me.
Just two questions:
- I did check the code, but did not read deep into formulas for each statistic, I assume they are correct. But if you want me to double check them, I can do that.
- What are the reference values for the results coming from? I guess Excel? I'm wondering if it would make sense to indicate somehow (in a very short way) how the references are computed if it is something that is not completely trivial, just in case someone wanted to double check them later if some issues were to appear.
| case MINIMUM -> start + 1; | ||
| case MAXIMUM -> index; | ||
| case DENSE -> dense; | ||
| case AVERAGE -> (start + 1 + index) / 2.0; |
There was a problem hiding this comment.
I'm just wondering out of curiosity in which situation one may want to use the average ranking?
(and why is it the default one? not saying it shouldn't be, just a honest question, because I haven't seen that)
Added test cases
Create helpers for promoting dataflow errors as panics. Remove set_at where possible.
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
jdunkerley
force-pushed
the
wip/jd/covariance-182059993
branch
from
df9297b
to
e3e75d4
Compare
hubertp
approved these changes
May 30, 2022
std-bits/base/src/main/java/org/enso/base/statistics/CorrelationStatistics.java
Show resolved
Hide resolved
jdunkerley
added a commit
that referenced
this pull request
May 31, 2022
- Added new `Statistic`s: Covariance, Pearson, Spearman, R Squared - Added `covariance_matrix` function - Added `pearson_correlation` function to compute correlation matrix - Added `rank_data` and Rank_Method type to create rankings of a Vector - Added `spearman_correlation` function to compute Spearman Rank correlation matrix # Important Notes - Added `Panic.throw_wrapped_if_error` and `Panic.handle_wrapped_dataflow_error` to help with errors within a loop. - Removed `Array.set_at` use from `Table.Vector_Builder`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
Statistics: Covariance, Pearson, Spearman, R Squaredcovariance_matrixfunctionpearson_correlationfunction to compute correlation matrixrank_dataand Rank_Method type to create rankings of a Vectorspearman_correlationfunction to compute Spearman Rank correlation matrixImportant Notes
Panic.throw_wrapped_if_errorandPanic.handle_wrapped_dataflow_errorto help with errors within a loop.Array.set_atuse fromTable.Vector_BuilderChecklist
Please include the following checklist in your PR:
Scala,
Java,
and
Rust
style guides.
./run.sh ide distand./run.sh ide watch.