[HigherOrderOp] add output tensor meta check for cond #113900
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/113900
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit f535862 with merge base 9cbee47 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR checks the tensor meta of the outputs of cond's branches. This helps us to identify several tests that return outputs that have different requires_grad. Also fix the error messages, which previously was in torch.ops.higher_order.cond now is raised in dynamo CondHigherOrder. Test Plan: Existing tests. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 aakhundov kadeng [ghstack-poisoned]
| # We check the meta data associated with meta["example_value"] | ||
| meta1 = _extract_tensor_metadata(var1.proxy.node.meta["example_value"]) | ||
| meta2 = _extract_tensor_metadata(var2.proxy.node.meta["example_value"]) | ||
| if meta1 != meta2: | ||
| all_diffs.append((f"pair{i}:", meta1, meta2)) |
There was a problem hiding this comment.
Does this also do strides? This might make cond difficult to use, but we can cross that bridge when it happens (e.g. if strides mismatch we can always just return .contiguous())
There was a problem hiding this comment.
It includes strides. Sounds good!
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: Command Details for Dev Infra teamRaised by workflow job |
This PR checks the tensor meta of the outputs of cond's branches. This helps us to identify several tests that return outputs that have different requires_grad. Also fix the error messages, which previously was in torch.ops.higher_order.cond now is raised in dynamo CondHigherOrder. Test Plan: Existing tests. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 aakhundov kadeng [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
3 participants
Stack from ghstack (oldest at bottom):
This PR checks the tensor meta of the outputs of cond's branches. This helps us to identify several tests that return outputs that have different requires_grad. Also fix the error messages, which previously was in torch.ops.higher_order.cond now is raised in dynamo CondHigherOrder.
Test Plan:
Existing tests.
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng