[Inductor] ReLU(Addmm) fusion

[nikitaved]

Stack from ghstack (oldest at bottom):

• [Inductor] GELU(Addmm) fusion #167998
• [Inductor] GELU: move unconditional decomp to fx_passes/post_grad #167997
• -> [Inductor] ReLU(Addmm) fusion #167469
• [Inductor] addmm with bias -> unfuse bias if there is a pointwise/reduction consumer #166165

Map ReLU(Addmm) pattern to _addmm_activation for [bias] + activation fusions.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben @mlazos

cc @eqy

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/167469

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 26 Pending, 2 Unrelated Failures

As of commit 8f31071 with merge base 1b43d6c ():

NEW FAILURES - The following jobs have failed:

• trunk / macos-py3-arm64 / test (default, 2, 3, macos-m1-stable) (gh)
  test/inductor/test_torchinductor.py::GPUTests::test_linear2_mps
• trunk / macos-py3-arm64 / test (default, 3, 3, macos-m1-stable) (gh)
  test/inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesGPUTests::test_linear2_dynamic_shapes_mps
• trunk / macos-py3-arm64 / test (mps, 1, 1, macos-m1-14) (gh)
  test/inductor/test_torchinductor.py::GPUTests::test_linear2_mps
• trunk / macos-py3-arm64 / test (mps, 1, 1, macos-m2-15) (gh)
  test/inductor/test_torchinductor.py::GPUTests::test_linear2_mps

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• inductor / unit-test / inductor-halide-test / test (inductor-halide, 1, 1, linux.12xlarge) (gh) (trunk failure)
  test/inductor/test_halide.py::HalideCpuTests::test__dyn_quant_pack_4bit_weight_bf16_cpu_halide

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

• trunk / linux-jammy-py3-clang12-executorch / test (executorch, 1, 1, lf.linux.2xlarge, unstable) (gh) (#166072)
  extension/llm/custom_ops/test_quantized_sdpa.py::SDPATestForCustomQuantizedSDPA::test_sdpa_with_custom_quantized_seq_len_small

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[nikitaved]

Decided to split ReLU and GELU -- GELU is being unconditionally decomposed which prevents capturing the pattern with the pattern matcher. Will address GELU in the follow-up.