[SPARK-7279] Removed diffSum which is theoretical zero in LinearRegression and coding formating #5809
Conversation
|
Jenkins, please test again. |
|
test this please |
| @@ -350,7 +345,6 @@ private class LeastSquaresAggregator( | |||
| } | |||
| } | |||
| lossSum += diff * diff / 2.0 | |||
| diffSum += diff | |||
There was a problem hiding this comment.
Basically you're saying this isn't read at all, which seems correct. I don't know if it's 0? (Haven't thought about it.) The rest looks like style changes which seems OK, but why remove the Array type, but keep the Long type, when you are making the literal a long explicitly?
There was a problem hiding this comment.
When I started to implement this code, I thought diffSum will not be zero. @mengxr caught that I don't add the correction terms back to the gradient correctly, but I still get the right models compared with R that made me go through the math again, and I found diffSum is always zero theoretically. Here is the proof added in the comment
15995c8
Array type is obvious since it's Array.ofDim. I did consider to remove the Long type, but in the end, I worry about someone can mistakenly remove L from 0L, so I decided to keep it.
|
Test build #31455 has finished for PR 5809 at commit
|
|
Test build #31456 has finished for PR 5809 at commit
|
|
LGTM. Merged into master. Thanks! |
…ssion and coding formating Author: DB Tsai <dbt@netflix.com> Closes apache#5809 from dbtsai/format and squashes the following commits: 6904eed [DB Tsai] triger jenkins 9146e19 [DB Tsai] initial commit
…ssion and coding formating Author: DB Tsai <dbt@netflix.com> Closes apache#5809 from dbtsai/format and squashes the following commits: 6904eed [DB Tsai] triger jenkins 9146e19 [DB Tsai] initial commit
…ssion and coding formating Author: DB Tsai <dbt@netflix.com> Closes apache#5809 from dbtsai/format and squashes the following commits: 6904eed [DB Tsai] triger jenkins 9146e19 [DB Tsai] initial commit
No description provided.