Skip to content

Commit

Permalink
[SPARK-6141][MLlib] Upgrade Breeze from 0.10 to 0.11 to fix convergen…
Browse files Browse the repository at this point in the history
…ce bug

LBFGS and OWLQN in Breeze 0.10 has convergence check bug.
This is fixed in 0.11, see the description in Breeze project for detail:

scalanlp/breeze#373 (comment)

Author: Xiangrui Meng <meng@databricks.com>
Author: DB Tsai <dbtsai@alpinenow.com>
Author: DB Tsai <dbtsai@dbtsai.com>

Closes apache#4879 from dbtsai/breeze and squashes the following commits:

d848f65 [DB Tsai] Merge pull request #1 from mengxr/AlpineNow-breeze
c2ca6ac [Xiangrui Meng] upgrade to breeze-0.11.1
35c2f26 [Xiangrui Meng] fix LRSuite
397a208 [DB Tsai] upgrade breeze
  • Loading branch information
mengxr committed Mar 4, 2015
1 parent d334bfb commit 76e20a0
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
2 changes: 1 addition & 1 deletion mllib/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
<dependency>
<groupId>org.scalanlp</groupId>
<artifactId>breeze_${scala.binary.version}</artifactId>
<version>0.10</version>
<version>0.11.1</version>
<exclusions>
<!-- This is included as a compile-scoped dependency by jtransforms, which is
a dependency of breeze. -->
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -372,8 +372,12 @@ class LogisticRegressionSuite extends FunSuite with MLlibTestSparkContext with M
testRDD2.cache()
testRDD3.cache()

val numIteration = 10

val lrA = new LogisticRegressionWithLBFGS().setIntercept(true)
lrA.optimizer.setNumIterations(numIteration)
val lrB = new LogisticRegressionWithLBFGS().setIntercept(true).setFeatureScaling(false)
lrB.optimizer.setNumIterations(numIteration)

val modelA1 = lrA.run(testRDD1, initialWeights)
val modelA2 = lrA.run(testRDD2, initialWeights)
Expand Down

0 comments on commit 76e20a0

Please sign in to comment.