Skip to content

Commit

Permalink
[SPARK-12811][ML] Estimator for Generalized Linear Models(GLMs)
Browse files Browse the repository at this point in the history
Estimator for Generalized Linear Models(GLMs) which will be solved by IRLS.

cc mengxr

Author: Yanbo Liang <ybliang8@gmail.com>

Closes apache#11136 from yanboliang/spark-12811.
  • Loading branch information
yanboliang authored and roygao94 committed Mar 22, 2016
1 parent 2635c41 commit 7a7a735
Show file tree
Hide file tree
Showing 4 changed files with 1,094 additions and 4 deletions.
Expand Up @@ -156,6 +156,12 @@ private[ml] class WeightedLeastSquares(

private[ml] object WeightedLeastSquares {

/**
* In order to take the normal equation approach efficiently, [[WeightedLeastSquares]]
* only supports the number of features is no more than 4096.
*/
val MAX_NUM_FEATURES: Int = 4096

/**
* Aggregator to provide necessary summary statistics for solving [[WeightedLeastSquares]].
*/
Expand All @@ -174,8 +180,8 @@ private[ml] object WeightedLeastSquares {
private var aaSum: DenseVector = _

private def init(k: Int): Unit = {
require(k <= 4096, "In order to take the normal equation approach efficiently, " +
s"we set the max number of features to 4096 but got $k.")
require(k <= MAX_NUM_FEATURES, "In order to take the normal equation approach efficiently, " +
s"we set the max number of features to $MAX_NUM_FEATURES but got $k.")
this.k = k
triK = k * (k + 1) / 2
count = 0L
Expand Down

0 comments on commit 7a7a735

Please sign in to comment.