Cannot implement TupleConverters for Higher-kinded Types

[laurencer]

Whilst you can implement a TupleConverter for a higher-kinded type (i.e. TupleConverter[ValidationNel[String, SomeCaseClass]]) - when used with a TypeDelimited pipe (i.e. TypedTsv[ValidationNel[String, SomeCaseClass]]) with multiple fields it will always throw NoSuchElementException: None.get (FileSource.scala:103).

This is because TypeDelimited will perform a rudimentary analysis to determine the classes to deserialize (see FileSource.scala:322 - 331). For a complex higher-kinded type, this will result in only a single class being assigned to types. If your input has multiple fields, this will throw an exception in DelimitedParser.java:136. This will then be caught in FileSource.scala:90 and discarded - making it very difficult to track down.

There are a number of ways this could be fixed - but they require non-trivial design changes. My thoughts are that the types should be manually specifiable in the TupleConverter (i.e. add some logic in TypeDelimited to check if the types have been provided by the TupleConverter).

This could be retrofitted fairly easily - by making types: Option[Seq[Class]] = None on TupleConverter.

I'm happy to write the fix - I just want to make sure I'm not missing something obvious.

[johnynek]

Isn't the problem just with the implementation of TypedDelimited?

I think there is a lot of deeper problem: in general how do go to and from strings without a separator?

We could require an Injection[T, String] to all the types inside?

[laurencer]

I guess there's a few different problems - I'm not fully across the code yet so not all of this may be correct:

1. We need a way to generate a String (with delimiters) from some input and vice versa.
2. We need a way to split that string (provided by TypedDelimited).
3. We need to be able to serialise/deserialise the types from each sub-string.
4. We need to have an Injection[T, (....)] where (...) is a tuple of some description.
5. And we need to offer error handling throughout this process (i.e. catch rows that don't serialise correctly, or which have the columns in the wrong order).

In my mind, TypedDelimited should handle 2. A TupleConverter should handle 3 and 4.

The TupleConverter then is composed of:

• An Injection[T, String] for each column in the source input (where T varies according to the specific data).
• An Injection[(...), U] where U is the final type of the tuple converter and (...) is the typed tuple of all the fields (obviously this doesn't actually have to be an Injection but could conceptually thats what needs to be included).

Thoughts?