Add registration API for torch.compile-eager

[EikanWang]

This PR is a follow-up of RFC #115545.

In this PR, we intend to provide a registration API dedicated to eager-through-torch.compile. The major workflow of this API will be as follows.

• Load cache
• Check cache according to the input tensors
  • Cache Hit: Run the cached kernel directly
  • Cache Miss: Run the AOTI to produce kernel and run the produced kernel. If AOTI fails to produce the kernel, invoke the python fallback function.

Currently, this PR always fallback to python kernel now and cache mechanism will be implemented in another PR - #116368

Stack from ghstack (oldest at bottom):

• [3/N] Non-Tensor: Support string parameter for aten operations #125831
• Separate AOTI Eager utils as a single file #125819
• Modularize aten parameter parser and checker #125308
• Support aten operations with out tensor #124926
• [2/N] Non-Tensor: Scalar Support: Add scalar to the cache for eager-through-torch.compile #124070
• [1/N] Non-Tensor: Scalar Support: Enable aot compile to support aten operations with scalar input like alpha #124177
• Add a cache mechanism to accelerate torch.compile-for-eager #116368
• -> Add registration API for torch.compile-eager #121387

cc @voznesenskym @penguinwu @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @amjames @desertfire @chauhang

Differential Revision: D57164385

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/121387

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (10 Unrelated Failures)

As of commit 56b60d7 with merge base 8cad88e ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 3, 5, linux.g5.4xlarge.nvidia.gpu) (gh) (similar failure)
  inductor/test_codecache.py::TestUtils::test_fresh_inductor_cache
• trunk / macos-12-py3-arm64 / test (default, 3, 3, macos-m1-stable) (gh) (similar failure)
  inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_multilayer_var_dynamic_shapes_cpu

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) (gh) (trunk failure)
  inductor/test_codecache.py::TestUtils::test_fresh_inductor_cache
• pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, linux.g5.4xlarge.nvidia.gpu) (gh) (trunk failure)
  inductor/test_max_autotune.py::TestMaxAutotune::test_max_autotune_remote_caching_dynamic_False
• pull / linux-focal-py3.11-clang10 / test (default, 1, 3, linux.2xlarge) (gh) (trunk failure)
  inductor/test_codecache.py::TestUtils::test_fresh_inductor_cache
• pull / linux-focal-py3.12-clang10 / test (default, 3, 3, linux.2xlarge) (gh) (trunk failure)
  inductor/test_codecache.py::TestUtils::test_fresh_inductor_cache
• pull / linux-focal-py3.8-clang10 / test (default, 2, 3, linux.2xlarge) (gh) (trunk failure)
  inductor/test_codecache.py::TestUtils::test_fresh_inductor_cache
• pull / linux-jammy-py3.8-gcc11 / test (default, 3, 3, linux.2xlarge) (gh) (trunk failure)
  inductor/test_codecache.py::TestUtils::test_fresh_inductor_cache
• trunk / macos-12-py3-arm64 / test (default, 2, 3, macos-m1-stable) (gh) (trunk failure)
  inductor/test_torchinductor.py::CpuTests::test_AllenaiLongformerBase_repro_cpu
• trunk / win-vs2019-cpu-py3 / test (default, 1, 3, windows.4xlarge.nonephemeral) (gh) (trunk failure)
  [ FAILED ] LoggingTest.ExceptionWhat

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[kit1980]

I need to revert this as it's failing internally

ERROR: test_torch_compile_override_registration_cuda (caffe2.test.inductor.test_torchinductor.GPUTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/dev/shm/uid-30083/32adf917-seed-nspid4026541534_cgpid12437421-ns-4026541785/caffe2/test/inductor/test_torchinductor.py", line 9819, in new_test
    return value(self)
  File "/dev/shm/uid-30083/32adf917-seed-nspid4026541534_cgpid12437421-ns-4026541785/caffe2/test/inductor/test_torchinductor.py", line 808, in test_torch_compile_override_registration
    res_array.append(getattr(torch, unary_op_name)(x))
RuntimeError: Error in dlopen: /re_tmp/tmpxk4fyq1k/cw74bdbcowopv7pivuo7czemlte5oieduln75gcpslbqehcdy7q2/cstf4iodqtmpabtysxkoezwgtt5d5nv6ooe6q4vaztopatoszyl7.so: undefined symbol: aoti_torch_device_type_cpu

I think the new file needs to be added to the buck dependencies.
@zou3519 @jansel see D56736862 if you want to help to re-land this.

[kit1980]

@pytorchbot revert -m "breaking internal builds" -c ghfirst

[pytorchmergebot]

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

[pytorchmergebot]

@EikanWang your PR has been successfully reverted.

[atalman]

@atalman has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[atalman]

@pytorchmergebot merge -f "Already landed in fbcode"

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here