[BEAM-4276] Surface graph.Fn encoding functions.#5620
Conversation
|
R: @herohde Oh that's exciting! gradle's correctly only running the Go Precommit :D |
herohde
left a comment
herohde
left a comment
There was a problem hiding this comment.
Is there a JIRA for what you're trying to do?
| } | ||
|
|
||
| // EncodeGraphFn encodes a *graph.Fn as a string. | ||
| func EncodeGraphFn(u *graph.Fn) (string, error) { |
There was a problem hiding this comment.
Are we not using these new helpers anywhere? The PR title suggests we're just making something existing public.
There was a problem hiding this comment.
It's related to Combiner Lifting, so I've used that jira tag.
Since it can help with writing a bundle level precombine, using existing combineFns, which is what I'm using it for at present, since the beam model/SDK doesn't yet support lifting combines through a CoGroupByKey. That's the root of it anyway. I like thinking of it as enabling the writing of MetaStructs.
The main goal is for me to avoid needing to write my precombine helper one way now and then re-writing later when there is official support for such a thing. If that feature does show up, I can simply use the same CombineFns over again.
Thanks!
lostluck
changed the title
|
R: @herohde PTAL Comments addressed. |
2 participants
This surfaces encoding functions for the higher level *graph.Fn representations. This permits users to serialize them for other uses, such as prototyping within bundle precombines within a ParDo. Types & functions should still be registered with beam within the binary for deserialization to produce a sensible result.