Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix local scoring with multipicklist features #243

Merged
merged 7 commits into from
Mar 20, 2019
Merged

fix local scoring with multipicklist features #243

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
47eb727
fix local scoring with multipicklist features
kinfaikan Mar 13, 2019
0978323
Merge branch 'master' into kk/local-multipicklist
tovbinm Mar 15, 2019
27f5a61
Merge branch 'master' of github.com:salesforce/TransmogrifAI into kk/…
kinfaikan Mar 19, 2019
c9a1232
address comments
kinfaikan Mar 19, 2019
cb8d81b
remove unnecessary import
kinfaikan Mar 19, 2019
f4a6718
Merge branch 'master' into kk/local-multipicklist
kinfaikan Mar 19, 2019
085bba6
Merge branch 'master' into kk/local-multipicklist
kinfaikan Mar 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions features/src/main/scala/com/salesforce/op/features/types/FeatureTypeSparkConverter.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,11 +192,11 @@ case object FeatureTypeSparkConverter {
// Maps
case wt if wt <:< weakTypeOf[t.MultiPickListMap] => (value: Any) =>
if (value == null) FeatureTypeDefaults.MultiPickListMap.value
else value.asInstanceOf[Map[String, MWrappedArray[String]]].map { case (k, v) => k -> v.toSet }
else value.asInstanceOf[Map[String, Seq[String]]].map { case (k, v) => k -> v.toSet }

// Sets
case wt if wt <:< weakTypeOf[t.MultiPickList] => (value: Any) =>
if (value == null) FeatureTypeDefaults.MultiPickList.value else value.asInstanceOf[MWrappedArray[String]].toSet
if (value == null) FeatureTypeDefaults.MultiPickList.value else value.asInstanceOf[Seq[String]].toSet
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you would also have to make the same fix for MultiPickListMap


// Everything else
case _ => identity
Expand Down
18 changes: 18 additions & 0 deletions features/src/test/scala/com/salesforce/op/features/types/FeatureTypeSparkConverterTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,13 @@ class FeatureTypeSparkConverterTest
Map("1" -> Seq(1.0, 5.0, 6.0)).toGeolocationMap -> Map("1" -> MWrappedArray.make(Array(1.0, 5.0, 6.0)))
)

val localMultiPickListValues = Table("mp",
(Seq("a", "b"), new MultiPickList(Seq("a", "b").toSet)),
(Nil, new MultiPickList(Set.empty[String])),
(null, new MultiPickList(Set.empty[String]))
)
val localMultiPickListMapValues = Seq(Map("k0" -> Seq("a", "b", "c")))

property("is a feature type converter") {
forAll(featureTypeConverters) { ft => ft shouldBe a[FeatureTypeSparkConverter[_]] }
}
Expand Down Expand Up @@ -204,4 +211,15 @@ class FeatureTypeSparkConverterTest
FeatureTypeSparkConverter.toSpark(featureValue) shouldEqual sparkValue
}
}
property("convert string seq to MultiPickList feature type") {
forAll(localMultiPickListValues) { case (local, expected) =>
FeatureTypeSparkConverter[MultiPickList]().fromSpark(local) shouldBe expected
}
}
property("convert multi-picklist map to MultiPickListMap feature type") {
localMultiPickListMapValues.foreach { local =>
val expected = new MultiPickListMap(local.map{ case (k, v) => k -> v.toSet })
FeatureTypeSparkConverter[MultiPickListMap]().fromSpark(local) shouldBe expected
}
}
}