Skip to content
This repository has been archived by the owner on Jan 20, 2022. It is now read-only.

Reducer estimation does not work with summingbird #713

Open
johnynek opened this issue Feb 9, 2017 · 0 comments
Open

Reducer estimation does not work with summingbird #713

johnynek opened this issue Feb 9, 2017 · 0 comments

Comments

@johnynek
Copy link
Collaborator

johnynek commented Feb 9, 2017

In scalding, Reducer estimation runs in this code:
https://github.com/twitter/scalding/blob/72ef68b465b17fcb56a29fc13ccb8b1013430b49/scalding-core/src/main/scala/com/twitter/scalding/ExecutionContext.scala#L91

but that code is not called in summingbird:

def toFlow(config: Config, timeSpan: Interval[Timestamp], mode: Mode, pf: PipeFactory[_]): Try[(Interval[Timestamp], Option[Flow[_]])] = {

so, the options are ignored.

if we solved #710 this would probably just go away.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant