JobTest with case class fails at runtime, while using a tuple works

[pzaborski]

This portion of code fails at runtime, with type Abc, while it works with (Abc, Abc).
The exception is NoSuchElementException (from None.get).

Code:

import com.twitter.scalding.{Args, Job, JobTest, TDsl, TypedTsv}
import org.specs.SpecificationWithJUnit

case class Abc(a: Int, b: Int)

object Type {
  type T =
    Abc
    //(Abc, Abc)       // it works with this type!
}

class MinimalJob(args: Args) extends Job(args) {
  import TDsl._
  import Type._

  TypedTsv[T]("input")
    .flatMap { x => Seq(x, x) }
    .write(TypedTsv[T]("output"))
}

class MinimalJobSpec extends SpecificationWithJUnit {
  import Type._

  "MinimalJob" should {
      JobTest(new MinimalJob(_))
      .source(TypedTsv[T]("input"), Seq.empty[T])
      .sink[T](TypedTsv[T]("output")) { buff =>
        buff.length must_== 0
      }
      .run
      .finish
  }
}

Exception:

org.specs.runner.UserError: java.util.NoSuchElementException: None.get
    at scala.None$.get(Option.scala:274)
    at scala.None$.get(Option.scala:272)
    at com.twitter.scalding.FileSource.createTap(FileSource.scala:103)
    at com.twitter.scalding.Source.read(Source.scala:90)
    at com.twitter.scalding.typed.TypedPipe$.from(TypedPipe.scala:42)
    at com.twitter.scalding.typed.TDsl$.mappableToTypedPipe(TDsl.scala:36)
    at MinimalJob.<init>(MinimalJobTest.scala:19)
    at MinimalJobSpec$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(MinimalJobTest.scala:28)
    at MinimalJobSpec$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(MinimalJobTest.scala:28)
    at com.twitter.scalding.JobTest.initJob(JobTest.scala:154)
    at com.twitter.scalding.JobTest.run(JobTest.scala:113)
    ...

[johnynek]

What version of scalding is this? Seems there is no .get in develop:

https://github.com/twitter/scalding/blob/develop/scalding-core/src/main/scala/com/twitter/scalding/FileSource.scala#L103

Also, you are hitting this issue:
https://github.com/twitter/scalding/blob/develop/scalding-core/src/main/scala/com/twitter/scalding/TypedDelimited.scala#L91

If you can suggest a way to fix it, that would be great. For now, the only work around is to do:

type T = Tuple1(Abc)

[johnynek]

It's a bit of a pain to get that right. We could check for equality with .erasure and Tuple1 to Tuple22. That might be more precise.

It's a bit of a hack (clearly) to map onto what cascading expects: runtime Class instances.

[johnynek]

https://github.com/twitter/scalding/blob/develop/scalding-core/src/main/scala/com/twitter/scalding/TypedDelimited.scala#L105

That is the incorrect logic now.

[pzaborski]

The version is 0.9.0rc1, the line is the same one as getOrElse in line 126 in develop:
https://github.com/twitter/scalding/blob/develop/scalding-core/src/main/scala/com/twitter/scalding/FileSource.scala#L126

Wrapping in Tuple1 helped on minimal version, thanks!