[Cutlass 3.3 submodule upgrade]

[kadeng]

Updates third_party/cutlass to Cutlass v3.3. No further changes appear neccessary.

Cutlass release 3.3 has not been tagged yet, the revision-hash is 1d7f2a207ec215e037099f4ba5632ccfa0249673 ( Cutlass 3.3 and two minor hotfixes on top )

Cutlass 3.3 offers the following improvements:

• Adds support for mixed precision GEMMs On Hopper and Ampere
• Adds support for < 16B aligned GEMMs on Hopper
• Enhancements to EVT
• Enhancements to Python interface
• Enhancements to Sub-byte type handling in CuTe
• Several other bug-fixes and performance improvements.
• minor doc update

Test Plan:

• CI ( ciflow/trunk, ciflow/inductor )
• pytest test/inductor/test_max_autotune.py

Stack from ghstack (oldest at bottom):

• [Inductor cutlass backend] Fixed workspace resize issue #115877
• [Inductor cutlass backend] Log CUDA compilation times #115814
• [Inductor cutlass backend] Added (some) doc comments and type annotations #115813
• [Inductor cutlass backend] Test and fix broadcasting of Aux / Bias edge cases #115655
• [Inductor cutlass backend] Robust Precompilation / Autotuning / Retuning in subprocesses #115654
• -> [Cutlass 3.3 submodule upgrade] #112861
• [Inductor cutlass backend] Support for StreamK and non-zero workspace sizes #114687
• [Inductor cutlass backend] Add support for row/column broadcasting of auxiliary inputs #115270
• [Inductor cutlass backend] Provide CUDA SM count to kernels #115174
• [Inductor cutlass backend] Generate standalone debug runners for Cutlass GEMMs #115072
• [inductor max autotune] Support sigmoid and tanh activations #114606
• [inductor max autotune] Flexible GEMM layout autotuning #114319
• [Inductor cutlass backend] Allow to fall back to non-fuseable ops #114075
• [Inductor cutlass backend] Experimental extended Cutlass op generator #113932
• [inductor max autotune] Log but continue on errors during autotuning benchmark #113891
• [Inductor Cutlass backend] Enabling flexible EVT-based pointwise fusions with additional tensor input #113959
• [Inductor cutlass backend] Enable torch.bmm and torch.baddbmm support #113890
• [Inductor cutlass backend] disable StreamK for GEMMs due to performance issues #113570
• [Cutlass inductor backend] Cutlass GEMM size threshold #113569
• [Inductor max autotune] Multithreaded Precompilation #113558
• [inductor max autotune] Detailed autotuning result logs ( machine-readable ) #113399
• [Inductor cutlass GEMM backend] Fix for operand memory layout change between autotuning and CUTLASSGEMMTemplate.render #113366
• [Inductor] Fix debug_str method of FusedSchedulerNode #113365

Differential Revision: D50988216

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @muchulee8 @aakhundov @ColinPeppler

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/112861

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 23 New Failures, 11 Unrelated Failures

As of commit bca0bd0 with merge base afe6d27 ():

NEW FAILURES - The following jobs have failed:

• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (cpu_inductor_torchbench, 1, 2, linux.12xlarge) (gh)
  hf_T5_base
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_inductor_timm, 1, 2, linux.12xlarge) (gh)
  mixer_b16_224
• Lint / lintrunner / linux-job (gh)
  >>> Lint for torch/_inductor/scheduler.py:
• pull / linux-docs / build-docs-python-false (gh)
  Process completed with exit code 2.
• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 2, 5, linux.4xlarge.nvidia.gpu) (gh)
  test_public_bindings.py::TestPublicBindings::test_modules_can_be_imported
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
  inductor/test_aot_inductor.py::AOTInductorTestABICompatibleCpu::test_dynamic_smem_above_default_limit_abi_compatible_cpu
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 2, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
  inductor/test_aot_inductor.py::AOTInductorTestABICompatibleCpu::test_addmm_multiple_dynamic_abi_compatible_cpu
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 3, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
  inductor/test_select_algorithm.py::TestSelectAlgorithm::test_mm_dup_args
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 5, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
  inductor/test_aot_inductor.py::AOTInductorTestABICompatibleCpuWithStackAllocationAndMinimalArrayRefInterface::test_addmm_multiple_dynamic_abi_compatible_cpu_with_stack_allocation_and_minimal_arrayref_interface
• pull / linux-focal-py3.11-clang10 / test (crossref, 1, 2, linux.2xlarge) (gh)
  test_public_bindings.py::TestPublicBindings::test_modules_can_be_imported
• pull / linux-focal-py3.11-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
  test_public_bindings.py::TestPublicBindings::test_modules_can_be_imported
• pull / linux-focal-py3.8-clang10 / test (crossref, 1, 2, linux.2xlarge) (gh)
  RuntimeError: inductor/test_debug_trace 1/1 failed
• pull / linux-focal-py3.8-clang10 / test (crossref, 2, 2, linux.2xlarge) (gh)
  RuntimeError: inductor/test_torchinductor 1/1 failed
• pull / linux-focal-py3.8-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
  RuntimeError: inductor/test_debug_trace 1/1 failed
• pull / linux-focal-py3.8-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
  RuntimeError: inductor/test_max_autotune 1/1 failed
• pull / linux-focal-py3.8-clang10 / test (default, 3, 3, linux.2xlarge) (gh)
  RuntimeError: inductor/test_torchinductor_dynamic_shapes 2/2 failed
• pull / linux-focal-py3.8-clang10 / test (dynamo, 2, 2, linux.2xlarge) (gh)
  Process completed with exit code 1.
• pull / linux-jammy-py3.10-clang15-asan / test (default, 2, 6, linux.4xlarge) (gh)
  test_public_bindings.py::TestPublicBindings::test_modules_can_be_imported
• pull / linux-jammy-py3.8-gcc11 / test (default, 1, 3, linux.2xlarge) (gh)
  RuntimeError: inductor/test_debug_trace 1/1 failed
• pull / linux-jammy-py3.8-gcc11 / test (default, 2, 3, linux.2xlarge) (gh)
  RuntimeError: inductor/test_max_autotune 1/1 failed
• pull / linux-jammy-py3.8-gcc11 / test (default, 3, 3, linux.2xlarge) (gh)
  RuntimeError: inductor/test_torchinductor_dynamic_shapes 2/2 failed
• pull / linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, linux.2xlarge) (gh)
  RuntimeError: distributed/test_inductor_collectives 1/1 failed
• pull / linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, linux.2xlarge) (gh)
  RuntimeError: distributed/_spmd/test_transformation 1/1 failed

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• .github/workflows/trunk.yml (gh)
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (cpu_inductor_huggingface, 1, 1, linux.12xlarge) (gh)
  YituTechConvBert
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (cpu_inductor_timm, 1, 2, linux.12xlarge) (gh)
  mixer_b16_224
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (cpu_inductor_timm, 2, 2, linux.12xlarge) (gh)
  xcit_large_24_p8_224
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (cpu_inductor_torchbench, 2, 2, linux.12xlarge) (gh)
  yolov3
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_inductor_huggingface, 1, 1, linux.12xlarge) (gh)
  YituTechConvBert
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_inductor_timm, 2, 2, linux.12xlarge) (gh)
  xcit_large_24_p8_224
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_inductor_torchbench, 1, 2, linux.12xlarge) (gh)
  hf_T5_base
• inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_inductor_torchbench, 2, 2, linux.12xlarge) (gh)
  yolov3
• pull / linux-focal-py3_8-clang9-xla / test (xla, 1, 1, linux.12xlarge) (gh)
  ModuleNotFoundError: No module named 'torch.version'
• pull / linux-focal-py3.8-clang10 / test (dynamo, 1, 2, linux.2xlarge) (gh)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[kadeng]

@kadeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[kadeng]

There is a bunch of CI errors, please take a look.

I was waiting for the official v3.3 tag. That's released now, and it actually fixes an important bug that (in the meantime ) caused build failures. Waiting for CI now..

[kadeng]

Moved to a (draft) feature branch, see #115919