[Easy][Profiler] Fix pattern matcher of profiler

[Aidyn-A]

Per title, as it fails with the following error if "+PTX" was used in TORCH_CUDA_ARCH_LIST:

  File "/usr/local/lib/python3.12/dist-packages/torch/profiler/_pattern_matcher.py", line 313, in skip
    has_tf32 = all(int(arch[3:]) >= 80 for arch in torch.cuda.get_arch_list())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/profiler/_pattern_matcher.py", line 313, in <genexpr>
    has_tf32 = all(int(arch[3:]) >= 80 for arch in torch.cuda.get_arch_list())
                   ^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: 'pute_120'

Because slicing arch[3:] will not end up on having only digits for compute_120 element of torch.cuda.get_arch_list():

>>> torch.cuda.get_arch_list()
['sm_75', 'sm_80', 'sm_86', 'sm_90', 'sm_100', 'sm_120', 'compute_120'] 

cc @ptrblck @msaroufim @eqy @jerryzh168 @robieta @chaekit @guotuofeng @guyang3532 @dzhulgakov @davidberard98 @briancoutinho @sraikund16 @sanrise

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/157711

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 42ce8d8 with merge base c5b46b5 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[sraikund16]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: 1 jobs have failed, first few of them are: trunk / macos-py3-arm64 / test (mps, 1, 1, macos-m2-15)

Details for Dev Infra team

Raised by workflow job

[sraikund16]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased fix_profiler_pattern_matcher onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout fix_profiler_pattern_matcher && git pull --rebase)

[sraikund16]

@Aidyn-A do you mind adding a test before merging this?

[Aidyn-A]

@Aidyn-A do you mind adding a test before merging this?

I do not think there is any need or way of adding short unit-tests. There are already a couple of them failing if PyTorch was built with TORCH_CUDA_ARCH_LIST="{any arches>=80} +PTX":

______________________________________________________________________________ TestExperimentalUtils.test_profiler_fp32_matmul_pattern ______________________________________________________________________________
Traceback (most recent call last):
  File "/usr/lib/python3.12/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/usr/lib/python3.12/unittest/case.py", line 634, in run
    self._callTestMethod(testMethod)
  File "/usr/lib/python3.12/unittest/case.py", line 589, in _callTestMethod
    if method() is not None:
       ^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/testing/_internal/common_utils.py", line 3146, in wrapper
    method(*args, **kwargs)
  File "/opt/pytorch/pytorch/test/profiler/test_profiler.py", line 2748, in test_profiler_fp32_matmul_pattern
    has_tf32 = 0 if pattern.skip else 1
                    ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/profiler/_pattern_matcher.py", line 313, in skip
    has_tf32 = all(int(arch[3:]) >= 80 for arch in torch.cuda.get_arch_list())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/profiler/_pattern_matcher.py", line 313, in <genexpr>
    has_tf32 = all(int(arch[3:]) >= 80 for arch in torch.cuda.get_arch_list())
                   ^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: 'pute_120'

To execute this test, run the following from the base repo dir:
    python test/profiler/test_profiler.py TestExperimentalUtils.test_profiler_fp32_matmul_pattern

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
__________________________________________________________________________ TestExperimentalUtils.test_profiler_pattern_matcher_json_report __________________________________________________________________________
Traceback (most recent call last):
  File "/usr/lib/python3.12/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/usr/lib/python3.12/unittest/case.py", line 634, in run
    self._callTestMethod(testMethod)
  File "/usr/lib/python3.12/unittest/case.py", line 589, in _callTestMethod
    if method() is not None:
       ^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/testing/_internal/common_utils.py", line 3146, in wrapper
    method(*args, **kwargs)
  File "/usr/local/lib/python3.12/dist-packages/torch/testing/_internal/common_utils.py", line 1617, in wrapper
    fn(*args, **kwargs)
  File "/opt/pytorch/pytorch/test/profiler/test_profiler.py", line 2885, in test_profiler_pattern_matcher_json_report
    report_all_anti_patterns(prof, json_report_dir=tmpdir, print_enable=False)
  File "/usr/local/lib/python3.12/dist-packages/torch/profiler/_pattern_matcher.py", line 629, in report_all_anti_patterns
    matched_events = anti_pattern.matched_events()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/profiler/_pattern_matcher.py", line 98, in matched_events
    if self.skip:
       ^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/profiler/_pattern_matcher.py", line 313, in skip
    has_tf32 = all(int(arch[3:]) >= 80 for arch in torch.cuda.get_arch_list())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/profiler/_pattern_matcher.py", line 313, in <genexpr>
    has_tf32 = all(int(arch[3:]) >= 80 for arch in torch.cuda.get_arch_list())
                   ^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: 'pute_120'

To execute this test, run the following from the base repo dir:
    python test/profiler/test_profiler.py TestExperimentalUtils.test_profiler_pattern_matcher_json_report

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

The reason this error was not regularly observed is because of Python interrupts the all call at the first False value. So this call is successful:

>>> all(int(arch[3:]) >= 80 for arch in ['sm_70', 'sm_80', 'sm_86', 'sm_90', 'sm_100', 'sm_120', 'compute_120'])
False

However, if all architectures are >=80, the Python interpreter checks the whole range and fails on compute_120:

>>> all(int(arch[3:]) >= 80 for arch in ['sm_80', 'sm_86', 'sm_90', 'sm_100', 'sm_120', 'compute_120'])
Traceback (most recent call last):
ValueError: invalid literal for int() with base 10: 'pute_120'

[sraikund16]

@Aidyn-A ok sounds good. Can you fix the linter error?

[Aidyn-A]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased fix_profiler_pattern_matcher onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout fix_profiler_pattern_matcher && git pull --rebase)

[Aidyn-A]

The linter was failing due to broken CI.

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here