-
Notifications
You must be signed in to change notification settings - Fork 21.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[functorch] jacrev: chunk_size=1 without vmap #91326
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/91326
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 FailuresAs of commit f6d92f7: BROKEN TRUNK - The following jobs failed but were present on the merge base 06bdd49:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor comment on edge case and testing and docs, but aside from that, this LGTM
@@ -492,9 +495,23 @@ def compute_jacobian_stacked(): | |||
for flat_basis_chunk in _chunked_standard_basis_for_(flat_output, | |||
flat_output_numels, | |||
chunk_size=chunk_size): | |||
if chunk_size == 1: | |||
flat_basis_chunk = tree_map(lambda t: torch.squeeze(t, 0), flat_basis_chunk) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should this be tree_map_only? (what happens for non-Tensor arguments?)
Also, squeeze is a bit dangerous, because if you squeeze on a dimension of size 2 it doesn't actually do anything. Would prefer an assertion that t.shape(0) == 1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These flat_basis_chunk
are guaranteed to be tensors
. Will add an assert.
Ref:
pytorch/torch/_functorch/eager_transforms.py
Line 615 in f62a3ca
def _chunked_standard_basis_for_(tensors, tensor_numels, chunk_size=None): |
Have addressed the reviews. PTAL @zou3519 :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thank you!
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: This PR is too stale; the last push date was more than 3 days ago. Please rebase and try again. You can rebase by leaving the following comment on this PR: Details for Dev Infra teamRaised by workflow job |
@pytorchbot merge -f"CI failure is unrelated" |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
As discussed at #91157 (comment)
cc @zou3519 @Chillee @samdow @soumith