[SPARK-13676] Fix mismatched default values for regParam in LogisticRegression

[dongjoon-hyun]

What changes were proposed in this pull request?

The default value of regularization parameter for LogisticRegression algorithm is different in Scala and Python. We should provide the same value.

Scala

scala> new org.apache.spark.ml.classification.LogisticRegression().getRegParam
res0: Double = 0.0

Python

>>> from pyspark.ml.classification import LogisticRegression
>>> LogisticRegression().getRegParam()
0.1

How was this patch tested?

manual. Check the following in pyspark.

>>> from pyspark.ml.classification import LogisticRegression
>>> LogisticRegression().getRegParam()
0.0

[srowen]

Jenkins test this please

[srowen]

Sounds good. If you have a moment would be good to look over other default regularization params to see if there are other mismatches.

[SparkQA]

Test build #52464 has finished for PR 11519 at commit d5323af.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[mengxr]

Merged into master. Thanks!

[dongjoon-hyun]

I see, @srowen . I will check the others, too.

Thank you, @srowen and @mengxr .

[dongjoon-hyun]

Also, thank you for Coverity, @srowen !
Today, I will check them, too. :)

[dongjoon-hyun]

Hi, @srowen .
According to your advice, I reviewed the other algorithms and I found that all other algorithms use the same default values for reqParam in Scala/Python. It's a good new.

• ALS: 0.1
• LassoWithSGD: 0.01
• LinearRegression: 0.0
• LinearRegressionWithSGD: 0.0
• LogisticRegression: 0.0
• LogisticRegressionWithSGD: 0.01
• LogisticRegressionWithLBFGS: 0.00
• RidgeRegressionWithSGD: 0.01
• SVMWithSGD: 0.01
• StreamingLogisticRegressionWithSGD: 0.0

However, I found that LinearRegressionWithSGD and StreamingLinearRegressionWithSGD does not have regParam as constructor arguments. They just depends on GradientDescent's default reqParam values. So, I think we need to file this as a new JIRA issue.

Thank you, @srowen .

[dongjoon-hyun]

Here is the JIRA Issue SPARK-13686 and PR.