Add DailySuffixTypedTsv and HourlySuffixTypedTsv. #873
Conversation
| @@ -42,6 +42,16 @@ class DailySuffixTsv(prefix: String, fs: Fields = Fields.ALL)(override implicit | |||
| override val fields = fs | |||
| } | |||
|
|
|||
| object DailySuffixTypedTsv { | |||
| def apply(prefix: String) | |||
There was a problem hiding this comment.
formatting is a little strange, I feel we usually prefer dropping the result to the next line before the arguments
| } | ||
|
|
||
| class DailySuffixTypedTsv(prefix: String)(override implicit val dateRange: DateRange) | ||
| extends DailySuffixSource(prefix, dateRange) with TypedSeperatedFile { |
There was a problem hiding this comment.
TypedSeparatedFile is not what makes something a TypedTsv. If you have this, what you will have is a DailySuffixSource which has an apply method which will create TypedTsv's. ie
(DailySuffixTypedTsv("prefix))("input) would be a TypedTsv. This is obviously not what you want. Waht you want is the thing that TypedSeparatedFile creates, a TypedDelimited
Does this make sense? TypedSeparatedFile is something which the TypedTsv, TypedCsv object etc extend for ease of the creation of various TypedDelimited. TypedDelimited is what makes something a TypedTsv. It's a little confusing I understand.
There was a problem hiding this comment.
I see, thanks for the clarification.
So extend TypedDelimited with DailySuffixSource should give me the desired DailySuffixTypedTsv I guess?
There was a problem hiding this comment.
Actually TypedDelimited is a class and extends FixedPathSource, so it won't work with DailySuffixSource. Does it make sense to change TypedDelimited to trait without extending any *PathSource? And create a FixedPathTypedDelimited class to replace the current TypedDelimited class?
|
That sounds pretty reasonable! |
Add FixedPathTypedDelimited to replace previous TypedDelimited class.
|
@jcoveney Mind taking another look at this? |
Add DailySuffixTypedTsv and HourlySuffixTypedTsv.
|
Thanks @reconditesea ! |
|
Thank you @jcoveney! |
#870
Directly use DailySuffixTypedTsv in SourceSpec tests.