Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
[WIP] JIT: Elmiminate SumToSize by using Optional Lists #18697
This PR is a proposed alternative to #18120 and would achieve very similar fusion (in particular for LSTM backward).
It consists of three parts:
Thus, in the LSTM case, no SumToSize remain in the crucial fusion group. The trick here is that we can specialize on the runtime information from the forward.
I label this WIP because I didn't integrate tests yet and because I didn't move all symbolic_script _grad_sum_to_size to the new logic.
However, it would be great to have some discussion about implementation details as they raised some eyebrows in #18407 (which did proposed specialization for Optiona[Tensor]) and its predecessor.
referenced this pull request
Apr 3, 2019
@ngimel pointed out a problem with this: The forward pass seems to instantiate intermediate results where previously we didn't. Thanks!
So after fixing this by taking passing the sizes, the forward performance is back and I could get rid of the bogus autodiff additions. But in a way, it's a bit of a mess that we now have broadcasting handling in autodiff and the graph_fuser that doesn't really interact (so e.g.
Hm. And merging the optional_None PR branch was a medium quality idea, because now we don't get the diff separately. The files changed relative to that are