[BEAM-3612] Closurize method invocations #7161
Conversation
|
|
||
| // RegisterStructWrapper takes in the reflect.Type of a structural DoFn, and | ||
| // a wrapping function that will take an instance of that struct type and | ||
| // produce a map of method names to of closured Funcs that call the method |
There was a problem hiding this comment.
... method names to closured Funcs...
| defer structFuncsMu.Unlock() | ||
|
|
||
| if t.Kind() != reflect.Struct { | ||
| log.Fatalf(context.Background(), "RegisterStructWrapper for %v should be a struct type, but was %v", t, t.Kind()) |
There was a problem hiding this comment.
Is this the extent of validity checking that can be performed on the struct? No required members?
There was a problem hiding this comment.
No, we could be more strict here. The generator does lifecycle method filtering so only the life cycle methods have code generated for them. Beam at pipeline construction time should be doing that required method checking at construction time anyway.
Short of always using the tool, the development flow would be using the reflection based paths, so one hopes that beam catches those errors then.
Finally, I'd rather have any more dedicated checks in a vetting runner of some kind, to catch typos and the like, rather than in the code generation code.
This wraps structuralDoFns in closures to avoid re-working the entire function analysis and binding stack. If we're going to generate code anyway, we may as well go all in.
Amends the benchmarks to include the "closurized" variation, and re-works the variable names to be (I hope) clearer than the previous iteration.
This is about the end of improving performance of StruturalDoFns invocation. Using the generation tool, it's possible to invoke methods ~20x faster than the reflective default path.
It's probably possible to do better by generating truly per function/per method specializations, rather than balancing with binary size, and always jump from the interface{} args straight to the actual function call, but I think we'd want to have that as an option in the shimx or starcgen packages, rather than do it all the time. That would permit users to balance binary size against speed.
Follow this checklist to help us incorporate your contribution quickly and easily:
[BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replaceBEAM-XXXwith the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.It will help us expedite review of your Pull Request if you tag someone (e.g.
@username) to look at it.Post-Commit Tests Status (on master branch)