Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Join fails due to type confusion for String #2

Closed
SteffenBach opened this issue Aug 3, 2021 · 1 comment
Closed

Join fails due to type confusion for String #2

SteffenBach opened this issue Aug 3, 2021 · 1 comment
Labels
bug Something isn't working

Comments

@SteffenBach
Copy link

The following example:

import com.audienceproject.crossbow.Implicits._

object Crossbow extends App {

  case class Score(name: String, points: Int)

  val scores = Seq(
    Score("abc", 20),
    Score("def", 15),
    Score("ghi", 10),
    Score("jkl", 5),
  ).flatMap(Score.unapply)
    .toDataFrame("name", "points")

  val names = Seq("abc", "jkl").toDataFrame("name")

  val filtered = scores.join(names, $"name")

  println(filtered.rowCount)
}

Fails like this:

Exception in thread "main" com.audienceproject.crossbow.exceptions.JoinException: name does not evaluate to the same type on both sides of the join.
	at com.audienceproject.crossbow.DataFrame.join(DataFrame.scala:198)
	at Crossbow$.delayedEndpoint$Crossbow$1(Crossbow.scala:17)
	at Crossbow$delayedInit$body.apply(Crossbow.scala:3)
	at scala.Function0.apply$mcV$sp(Function0.scala:39)
	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
	at scala.App.$anonfun$main$1(App.scala:76)
	at scala.App.$anonfun$main$1$adapted(App.scala:76)
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:919)
	at scala.App.main(App.scala:76)
	at scala.App.main$(App.scala:74)
	at Crossbow$.main(Crossbow.scala:3)
	at Crossbow.main(Crossbow.scala)

The column type of scores.name resolves to scala.Predef.String whereas names.name resolves to java.lang.String causing the strict comparison to evaluate to false.

@jacobfi jacobfi added the bug Something isn't working label Aug 3, 2021
@jacobfi
Copy link
Contributor

jacobfi commented Dec 13, 2022

Fixed in fa2f685

@jacobfi jacobfi closed this as completed Dec 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

2 participants