PrimTorch + nvFuser backend #1281
Conversation
benchmarks/common.py
Outdated
| parser.add_argument( | ||
| "--prims-nvfuser", action="store_true", help="user prims + nvfuser backend" | ||
| ) | ||
| parser.add_argument( |
There was a problem hiding this comment.
Nit - We can directly use --backend=nvprims_nvfuser, if you didn't know about this option.
There was a problem hiding this comment.
I didn't know! I will try and remove this code then. Yep, it works, meaning we'd need to modify our scripts 😄
|
|
||
| # prims_nvfuser uses the prims and AOT-Autograd to get FX-aten IR. And then | ||
| # directly lowers to NVFuser without relying no Torchscript. | ||
| BACKENDS["prims_nvfuser"] = aot_prims_nvfuser |
There was a problem hiding this comment.
Is this backend still needed? What is the difference with nvprims_nvfuser?
There was a problem hiding this comment.
It's not needed for our team and there are no plans to use it or recommend anyone to use it. It wasn't me who added this, so I'm not going to touch it.
"prims_nvfuser" was added in #584 and it makes a decision of translating aten->prims based on a static outdated and not maintained table from https://github.com/pytorch/pytorch/blob/09965957cd8ecc696852e73022892b3ad4475783/torch/fx/passes/backends/nvfuser.py#L70-L71
The performance of this backend is terrible, on the order of 5% of eager mode's performance.
"nvprims_nvfuser" decomposes aten ops into nvprims, a subset of prims that are guaranteed to be executable by nvFuser, and the decomposition is used only if given aten op is fully decomposable into nvprims.
There was a problem hiding this comment.
Let's just remove it if you don't recommend anyone use it.
There was a problem hiding this comment.
Maybe Sherlock would like to continue working on his approach. Can we handle this separately and not remove the "prims_nvfuser" backend in this PR?
There was a problem hiding this comment.
I created this backend for your team's use.
Now that you have your own implementation, please feel free to remove this.
|
@anijain2305 I reverted changes to |
|
Can someone hit the merge button? |
|
You can run |
|
@desertfire, thanks, I ran it. Could you please approve running of the workflows? |
|
Thanks @IvanYashchuk for the contribution. Your PR is merged. |
Removing "prims_nvfuser" backend according to the discussion in pytorch/torchdynamo#1281 (comment). cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx Pull Request resolved: #88083 Approved by: https://github.com/ezyang
Removing "prims_nvfuser" backend according to the discussion in pytorch/torchdynamo#1281 (comment). cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx Pull Request resolved: pytorch#88083 Approved by: https://github.com/ezyang
Removing "prims_nvfuser" backend according to the discussion in pytorch/torchdynamo#1281 (comment). cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx Pull Request resolved: pytorch#88083 Approved by: https://github.com/ezyang
6 participants
This PR adds a new AOT Autograd-based backend with the "--backend=nvprims_nvfuser" cli argument for benchmarks.