Skip to content
This repository has been archived by the owner on Nov 11, 2022. It is now read-only.

Aggregators cannot be created dynamically #55

Open
francesperry opened this issue Sep 9, 2015 · 2 comments
Open

Aggregators cannot be created dynamically #55

francesperry opened this issue Sep 9, 2015 · 2 comments

Comments

@francesperry
Copy link
Contributor

Currently, Aggregators have to be created during DoFn construction. That rules out useful cases like dynamically creating a small number of aggregators to track exceptions that get thrown during DoFn execution.

(See http://stackoverflow.com/questions/32469446/create-aggregators-dynamically-for-exceptions )

@dovy
Copy link

dovy commented Nov 3, 2015

+1

Or at least key/value based aggregator, so we can group and add according to key. So we could log each "type" of exception via key or something like that.

@scattamanchi
Copy link

I have latest version of Cloud SDK. I submitted a job on Google's Cloud DataFlow and I don't see it Aggregators getting created dynamically during execution. Did the fix get into Google Cloud DataFlow?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants