Fold operation

[idubinskiy]

Is there any way to implement a fold operation with the current dataset methods? If not, is there a plan to add it? I'd be willing to give it a shot.

[chrislusf]

You are welcome to hack this!

Do you have any specific needs for this? I was thinking just use Go's default value in reduce operation, to avoid the need for fold.

Fold can not be parallelized.

http://stackoverflow.com/questions/25158780/difference-between-reduce-and-foldleft-fold-in-functional-programming-particula

[idubinskiy]

just use Go's default value in reduce operation

Not sure what you mean by that.

As far as specific need, I'm trying to re-implement an algorithm using Glow, and ran into a bit of a roadblock. But I think I may be able to work around it by doing Map(type A -> type B).Reduce(typeB, type B -> type B), or maybe ReduceByKey.

[chrislusf]

I see. That'll work.

And let me know if glow works for you or not.

I would prefer not to implment fold, which can not be done efficiently.