Side inputs not working in CombineGlobally

[damccorm]

Side inputs are not working in CombineGlobally. They do work as expected in CombinePerKey and CombineValues.

 

The function argument still has the value of AsSingleton rather than being resolved into the real value.

 

Here is a Notebook where you can reproduce the issue.

https://colab.research.google.com/drive/149By0ZKJjb_JdDOsFywdT_OLj1hMEPBa

Imported from Jira BEAM-8400. Original Jira may contain additional context.
Reported by: dcavazos.

[tvalentyn]

From mailing list thread a while back (https://lists.apache.org/thread/zbzzd1w9wjcdbff6nppjp2o079x1xng8):

You can work around this by specifying
without_defaults() on the global combine (as the default is computed
at pipeline construction time). Note that even in the cases where it
works, it disables combiner lifting, so side inputs in combiners is
generally discouraged.

[hjtran]

I haven't put much thought into whether this will work with windowed pcollections. If not, maybe we can just have this implement side inputs for the batch case and error for the streaming case and punt on the streaming case