[SPARK-5891][ML] Add Binarizer ML Transformer

[viirya]

JIRA: https://issues.apache.org/jira/browse/SPARK-5891

[SparkQA]

Test build #30952 has finished for PR 5699 at commit 1682f8c.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.
• This patch does not change any dependencies.

[viirya]

The failure is caused by an unrelated test in streaming.

[viirya]

please retest.

[mengxr]

test this please

[mengxr]

There is a problem with HasThreshold. Because in the doc we said "threshold used in binary classification". Maybe we should implement `threshold" param in Binarizer and document it correctly. Also, we need to document what the output is if the input equals the threshold.

[SparkQA]

Test build #30972 has finished for PR 5699 at commit 1682f8c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.
• This patch does not change any dependencies.

[SparkQA]

Test build #31124 has finished for PR 5699 at commit cc4f03c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • final class Binarizer extends Transformer with HasInputCol with HasOutputCol
  • trait ExpectsInputTypes
  • abstract class BinaryMathExpression(f: (Double, Double) => Double, name: String)
  • case class Pow(left: Expression, right: Expression) extends BinaryMathExpression(math.pow, "POWER")
  • case class Hypot(
  • case class Atan2(
  • abstract class MathematicalExpression(name: String)
  • abstract class MathematicalExpressionForDouble(f: Double => Double, name: String)
  • abstract class MathematicalExpressionForInt(f: Int => Int, name: String)
  • abstract class MathematicalExpressionForFloat(f: Float => Float, name: String)
  • abstract class MathematicalExpressionForLong(f: Long => Long, name: String)
  • case class Sin(child: Expression) extends MathematicalExpressionForDouble(math.sin, "SIN")
  • case class Asin(child: Expression) extends MathematicalExpressionForDouble(math.asin, "ASIN")
  • case class Sinh(child: Expression) extends MathematicalExpressionForDouble(math.sinh, "SINH")
  • case class Cos(child: Expression) extends MathematicalExpressionForDouble(math.cos, "COS")
  • case class Acos(child: Expression) extends MathematicalExpressionForDouble(math.acos, "ACOS")
  • case class Cosh(child: Expression) extends MathematicalExpressionForDouble(math.cosh, "COSH")
  • case class Tan(child: Expression) extends MathematicalExpressionForDouble(math.tan, "TAN")
  • case class Atan(child: Expression) extends MathematicalExpressionForDouble(math.atan, "ATAN")
  • case class Tanh(child: Expression) extends MathematicalExpressionForDouble(math.tanh, "TANH")
  • case class Ceil(child: Expression) extends MathematicalExpressionForDouble(math.ceil, "CEIL")
  • case class Floor(child: Expression) extends MathematicalExpressionForDouble(math.floor, "FLOOR")
  • case class Rint(child: Expression) extends MathematicalExpressionForDouble(math.rint, "ROUND")
  • case class Cbrt(child: Expression) extends MathematicalExpressionForDouble(math.cbrt, "CBRT")
  • case class Signum(child: Expression) extends MathematicalExpressionForDouble(math.signum, "SIGNUM")
  • case class ISignum(child: Expression) extends MathematicalExpressionForInt(math.signum, "ISIGNUM")
  • case class FSignum(child: Expression) extends MathematicalExpressionForFloat(math.signum, "FSIGNUM")
  • case class LSignum(child: Expression) extends MathematicalExpressionForLong(math.signum, "LSIGNUM")
  • case class ToDegrees(child: Expression)
  • case class ToRadians(child: Expression)
  • case class Log(child: Expression) extends MathematicalExpressionForDouble(math.log, "LOG")
  • case class Log10(child: Expression) extends MathematicalExpressionForDouble(math.log10, "LOG10")
  • case class Log1p(child: Expression) extends MathematicalExpressionForDouble(math.log1p, "LOG1P")
  • case class Exp(child: Expression) extends MathematicalExpressionForDouble(math.exp, "EXP")
  • case class Expm1(child: Expression) extends MathematicalExpressionForDouble(math.expm1, "EXPM1")
• This patch does not change any dependencies.

[mengxr]

Only HasInputCol and HasOutputCol are used. So this could be more explicit.

[viirya]

@mengxr updated. Thanks for comments.

[SparkQA]

Test build #31550 has finished for PR 5699 at commit 1a0b9a4.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • final class Binarizer extends Transformer with HasInputCol with HasOutputCol

[viirya]

retest this please.

[SparkQA]

Test build #31553 has finished for PR 5699 at commit 1a0b9a4.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • final class Binarizer extends Transformer with HasInputCol with HasOutputCol

[mengxr]

minor: data could be a val

[mengxr]

LGTM. Merged into master. Thanks!