Typedpipe partition #987
Typedpipe partition #987
Conversation
Merge from twitter/scalding
|
Looks like RichPipe has a |
|
well, I think we should keep TypedPipe as close as possible to scala.Iterable or Seq, so I think it is fine. It was probably a mistake in RichPipe, but oh well. Not all things are named the same between the two. LGTM. Merge when green. |
|
Makes sense. Travis seems to be having some trouble pulling down deps :(
I'll retry after a while I guess. |
|
change the http deps to https. We saw this earlier today. On Thu, Jul 31, 2014 at 1:43 PM, Ruban Monu notifications@github.com
Oscar Boykin :: @posco :: http://twitter.com/posco |
| * | ||
| * Sometimes what you really want is a groupBy in these cases. | ||
| */ | ||
| def partition(p: T => Boolean): (TypedPipe[T], TypedPipe[T]) = { |
There was a problem hiding this comment.
We may need some rules of thumb of when to use this V.S when to use Group operation.
There was a problem hiding this comment.
I think it would be partition when you want to split your 'dataflow' into two, groupBy when you want to create a grouping in the same dataflow?
Anything else that comes to mind? I'm still partially hung over in the fields API world so might be missing something.
(TypedPipe.groupBy does not exactly match scala Iterable.groupBy, gives you a Grouped and not a Map[K, TypedPipe[T]. If that were the case, then partition would be the same as groupBy on a boolean which gives you (TypedPipe[T], TypedPipe[T]) instead of a Map[Boolean, TypedPipe[T]].)
|
merge when green. |
|
closes #986 |
Addresses #986