[Re-landing 68111] Add JIT graph fuser for oneDNN Graph API (Preview4.1)

[sanchitintel]

Description

Relanding #68111
Preview4 PR of this RFC.

On the basis of #50256, the below improvements are included:

• The preview4 release branch of the oneDNN Graph API is used
• The fuser now works with the profiling graph executor. We have inserted type check nodes to guard the profiled tensor properties.

User API:

The optimization pass is disabled by default. Users could enable it by:

torch.jit.enable_onednn_fusion(True)

Performance:

pytorch/benchmark tool is used to compare the performance:

• SkyLake 8180 (1 socket of 28 cores):
  
• SkyLake 8180 (single thread):
  
  • By mapping hardswish to oneDNN Graph, it’s 8% faster than PyTorch JIT (NNC + OFI)
    ** We expect performance gain after mapping transpose, contiguous & view to oneDNN graph ops

Directory structure of the integration code

Fuser-related code are placed under:

torch/csrc/jit/codegen/onednn/

Optimization pass registration is done in:

torch/csrc/jit/passes/onednn_graph_fuser.h

CMake for the integration code is:

caffe2/CMakeLists.txt

Limitations

• In this PR, we have only supported the optimization on Linux platform. The support on Windows and MacOS will be enabled as a next step.
• We have only optimized the inference use case.

[facebook-github-bot]

🔗 Helpful links

• 🧪  See artifacts and rendered test results at hud.pytorch.org/pr/74572
• Need help or want to give feedback on the CI? Visit our office hours

💊 CI failures summary and remediations

As of commit c02ece3 (more details on the Dr. CI page):

---

• 4/4 failures introduced in this PR

---

4 failures not recognized by patterns:

Job	Step	Action
pull / linux-xenial-cuda11.3-py3.7-gcc7 / build	Unknown	🔁 rerun
pull / linux-bionic-rocm4.5-py3.7 / build	Unknown	🔁 rerun
pull / linux-xenial-py3.7-gcc5.4-mobile-lightweight-dispatch-build / build	Unknown	🔁 rerun
pull / linux-vulkan-bionic-py3.7-clang9 / build	Unknown	🔁 rerun

---

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[malfet]

@sanchitintel to make review easier, can you simply cherry-picked landed commit into the branch and then apply any other changes on top of that?

[sanchitintel]

to make review easier, can you simply cherry-picked landed commit into the branch and then apply any other changes on top of that?

Sorry @malfet, please clarify which landed commit you're referring to.
Please confirm if you mean first rebasing PR #68111 with the master branch, and then adding this commit to fix the lite-interpreter build. Thanks!

[malfet]

This one: cd17683

[sanchitintel]

Windows build failed while compiling a lite interpreter file (test_jit/CMakeFiles/test_jit.dir/test_lite_interpreter.cpp.obj), but seems to have failed due to an unrelated cause -

Will rebase later to check if the issue got fixed.
Similar failures in other PRs, such as #74586.

[sanchitintel]

Closing & reopening as #74596. Thanks!

[sanchitintel]

Somehow CI is not running on #74596. GitHub Actions outage is over