cached output allocation is broken #2002
Assignees
Comments
This was referenced Sep 28, 2022
Merged
jjsjann123
added a commit
that referenced
this issue
Sep 30, 2022
Fixes #2002 checks all IterDomain on outputs and disables verifies that no extent value is a consumer of fusion inputs.
pytorchmergebot
pushed a commit
to pytorch/pytorch
that referenced
this issue
Oct 10, 2022
Cherry-picked from devel branch: csarofeen#2010 turns on accidentally disabled output allocation cache [#2002](csarofeen#2002) Updated check for safety regarding allocation cache by iterating all IterDomain on outputs and enables cache re-use only when no extent value is a consumer of fusion inputs (output sizes is not dependent on scalar inputs). Pull Request resolved: #86100 Approved by: https://github.com/csarofeen
jjsjann123
added a commit
to jjsjann123/nvfuser
that referenced
this issue
Oct 29, 2022
Cherry-picked from devel branch: csarofeen/pytorch#2010 turns on accidentally disabled output allocation cache [#2002](csarofeen/pytorch#2002) Updated check for safety regarding allocation cache by iterating all IterDomain on outputs and enables cache re-use only when no extent value is a consumer of fusion inputs (output sizes is not dependent on scalar inputs). Pull Request resolved: pytorch/pytorch#86100 Approved by: https://github.com/csarofeen
jjsjann123
added a commit
to jjsjann123/nvfuser
that referenced
this issue
Nov 10, 2022
Cherry-picked from devel branch: csarofeen/pytorch#2010 turns on accidentally disabled output allocation cache [#2002](csarofeen/pytorch#2002) Updated check for safety regarding allocation cache by iterating all IterDomain on outputs and enables cache re-use only when no extent value is a consumer of fusion inputs (output sizes is not dependent on scalar inputs). Pull Request resolved: pytorch/pytorch#86100 Approved by: https://github.com/csarofeen
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
馃悰 Describe the bug
FusionExecutor caches output sizes. e.g. for the same input sizes, we'll reuse the cached output sizes instead of running evaluation on outputs again.
This doesn't work any more with our recent expansion on codegen (TensorFactory methods & RNG ops). Unfortunately the issue was caught earlier, since the caching re-use gets accidentally disabled in my refactor code. https://github.com/csarofeen/pytorch/pull/1914/files#diff-3e62c8296c8362cd8c14a3d3300e5b2758d09b163ade856eefbf7361d75d7acaR373
So I'm going to add a check in executor and walk through fusion ops and disable cache re-use when I see those ops. I think there should be a more robust/aggressive way to handle this. -> we only needed to disable the cached output allocation when the factory methods depends on runtime scalar input.
Versions
ToT devel
The text was updated successfully, but these errors were encountered: