Math transformers part 2 of 2 #257
Conversation
…ifAI into lm/mathTransformers
…mathTransformers2
Codecov Report
@@ Coverage Diff @@
## master #257 +/- ##
==========================================
+ Coverage 86.61% 86.63% +0.02%
==========================================
Files 315 315
Lines 10345 10366 +21
Branches 346 546 +200
==========================================
+ Hits 8960 8981 +21
Misses 1385 1385
Continue to review full report at Codecov.
|
| uid: String = UID[FloorTransformer[_]] | ||
| )( | ||
| implicit override val tti: TypeTag[I] | ||
| ) extends UnaryTransformer[I, Integral](operationName = "floor", uid = uid){ |
core/src/main/scala/com/salesforce/op/stages/impl/feature/MathTransformers.scala
Outdated
Show resolved
Hide resolved
core/src/main/scala/com/salesforce/op/stages/impl/feature/MathTransformers.scala
Outdated
Show resolved
Hide resolved
core/src/main/scala/com/salesforce/op/stages/impl/feature/MathTransformers.scala
Outdated
Show resolved
Hide resolved
core/src/main/scala/com/salesforce/op/stages/impl/feature/MathTransformers.scala
Outdated
Show resolved
Hide resolved
| f2.originStage.isInstanceOf[AbsoluteValueTransformer[_]] shouldBe true | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
please remove redundant trailing endlines in all test files
core/src/test/scala/com/salesforce/op/stages/impl/feature/AbsoluteValueTransformerTest.scala
Outdated
Show resolved
Hide resolved
| val sample = Seq(Real(-1.3), Real(-4.9), Real.empty, Real(5.1), Real(-5.1), Real(0.1), Real(2.5), Real(0.4)) | ||
| val (inputData, f1) = TestFeatureBuilder(sample) | ||
| val transformer: ExpTransformer[Real] = new ExpTransformer[Real]().setInput(f1) | ||
| override val expectedResult: Seq[Real] = Seq(Real(math.exp(-1.3)), Real(math.exp(-4.9)), Real.empty, |
There was a problem hiding this comment.
Seq(..).map(math.exp).map(_.toReal)
| val (inputData, f1) = TestFeatureBuilder(sample) | ||
| val transformer: PowerTransformer[Real, Double] = new PowerTransformer[Real, Double](3.0).setInput(f1) | ||
| override val expectedResult: Seq[Real] = Seq(Real(math.pow(-1.3, 3)), Real(math.pow(-4.9, 3)), Real.empty, | ||
| Real(math.pow(5.1, 3)), Real(math.pow(-5.1, 3)), Real(math.pow(0.1, 3)), Real(math.pow(2.5, 3)), |
There was a problem hiding this comment.
same here Seq(...).map(math.pow).map(_.toReal)
core/src/main/scala/com/salesforce/op/stages/impl/feature/MathTransformers.scala
Outdated
Show resolved
Hide resolved
…rifAI into lm/mathTransformers2
|
@leahmcguire one thought I had after I merged the PR is that for features of more concrete numeric types, i.e. class ExpTransformer[I <: Real]
(
uid: String = UID[ExpTransformer[_]]
)(
implicit override val tti: TypeTag[I]
) extends UnaryTransformer[I, I](operationName = "exp", uid = uid){
private val convert = FeatureTypeSparkConverter[I]()
override def transformFn: I => I = (i: I) => convert.fromSpark(i.toDouble.map(math.exp).filter(Number.isValid).orNull)
} |
|
Well I'm not sure that the exp (or log or power) of currency is still currency. I'm sure percent is not still a percent. The RealNN is the only one that it may not be reasonable to downgrade and the difference in treatment between the Real and RealNN is just the null check and fill - which would have no effect if there are no nulls... |
|
Thanks for the contribution! Unfortunately we can't verify the commit author(s): leahmcguire <l***@s***.com> Leah McGuire <l***@s***.com>. One possible solution is to add that email to your GitHub account. Alternatively you can change your commits to another email and force push the change. After getting your commits associated with your GitHub account, refresh the status of this Pull Request. |
Related issues
Made transformers and shortcuts for common math operations