New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-9005][MLlib]Fix RegressionMetrics computation of explainedVariance #7361
[SPARK-9005][MLlib]Fix RegressionMetrics computation of explainedVariance #7361
Conversation
Test build #37104 has finished for PR 7361 at commit
|
Test build #37106 has finished for PR 7361 at commit
|
Test build #37109 has finished for PR 7361 at commit
|
Test build #37118 has finished for PR 7361 at commit
|
val yMean = summary.mean(0) | ||
predictionAndObservations.map { | ||
case (prediction, _) => math.pow(prediction - yMean, 2) | ||
}.reduce(_ + _) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.sum
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Test build #37140 has finished for PR 7361 at commit
|
jenkins retest this please |
Test build #37149 has finished for PR 7361 at commit
|
* explainedVariance = 1 - variance(y - \hat{y}) / variance(y) | ||
* Reference: [[http://en.wikipedia.org/wiki/Explained_variation]] | ||
* Returns the variance explained by regression. | ||
* @see [[https://en.wikipedia.org/wiki/Fraction_of_variance_unexplained]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you put the formula here please?
LGTM pending tests (except for missing doc) |
Test build #37174 has finished for PR 7361 at commit
|
LGTM |
Test build #1070 has finished for PR 7361 at commit
|
@feynmanliang Could you please modify the title and description so they just say that explainedVariance is being fixed? (r2 is not changing, right?) |
Merging with master |
Fixes implementation of
explainedVariance
andr2
to be consistent with their definitions as described in SPARK-9005.