[Kineto] Improve Config Options for Input Shapes, Memory, Stack, Flops, and Modules - Part 1

[aaronenyeshi]

Summary:
Improve On-Demand Kineto config to enable toggling of profiler options via the config file. New config strings:

• PROFILE_REPORT_INPUT_SHAPES
• PROFILE_PROFILE_MEMORY
• PROFILE_WITH_STACK
• PROFILE_WITH_FLOPS
• PROFILE_WITH_MODULES

Also marked for deprecation, but still valid, old config options:

• CLIENT_INTERFACE_ENABLE_OP_INPUTS_COLLECTION
• PYTHON_STACK_TRACE

Test Plan: CI Tests (internal testing)

Reviewed By: leitian

Differential Revision: D44275220

Pulled By: aaronenyeshi

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/97380

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 Failures

As of commit 56640f4:

NEW FAILURES - The following jobs have failed:

• linux-bionic-cuda11.8-py3.10-gcc7 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) (gh)
• linux-focal-rocm5.4.2-py3.8 / test (default, 1, 3, linux.rocm.gpu) (gh)
• linux-focal-rocm5.4.2-py3.8 / test (default, 2, 3, linux.rocm.gpu) (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D44275220

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D44275220

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D44275220

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D44275220

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D44275220

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D44275220

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D44275220

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D44275220

[davidberard98]

mirror internal stamp 👍

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D44275220

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D44275220

[facebook-github-bot]

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[qqaatw]

Hi @aaronenyeshi some changes this PR brought caused a compilation error on my system:

pytorch/torch/csrc/profiler/kineto_client_interface.cpp:30:34: error: only virtual member functions can be marked 'override'
      bool with_modules = false) override {
                                 ^~~~~~~~~

PyTorch version: (viable/strict)
Is debug build: True
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.6.1 (arm64)
GCC version: Could not collect
Clang version: 13.1.6 (clang-1316.0.21.2.5)
CMake version: version 3.23.2
Libc version: N/A

Python version: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:00:33)  [Clang 13.0.1 ] (64-bit runtime)
Python platform: macOS-12.6.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: False

Versions of relevant libraries:
[pip3] mypy==0.960
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.1
[pip3] torch==2.1.0a0+git38443eb
[pip3] torchvision==0.15.1
[conda] numpy                     1.23.1                   pypi_0    pypi
[conda] torch                     2.1.0a0+git38443eb           dev_0    <develop>
[conda] torchvision               0.15.1                   pypi_0    pypi

cc @davidberard98 and @ezyang - Should we add a workflow job that tests on newer Clang compilers (I guess this is the cause)? previous incidents that broke the builds on my system: #96878 (comment) #92755 (comment)

[ezyang]

upgrading clang seems reasonable; I'm surprised we're not getting this error already though. Can you send the (hopefully easy) patch up?

[ezyang]

cc @malfet

[malfet]

@ezyang, We have recent enough clang(and even gcc) in the system, but it does not protect one from a missing submodule update (which introduced this function)

Just to test it still works: #97782

[malfet]

Hi @aaronenyeshi some changes this PR brought caused a compilation error on my system:

pytorch/torch/csrc/profiler/kineto_client_interface.cpp:30:34: error: only virtual member functions can be marked 'override'
      bool with_modules = false) override {
                                 ^~~~~~~~~

@qqaatw hmm, did you forget to run git submodule update? As prepare is indeed virtual function
See https://github.com/pytorch/kineto/blob/e64df4dc31285a6129a74d26d67365cedf7aa6d1/libkineto/include/ClientInterface.h#L17

[qqaatw]

@malfet Yeah I forgot, my fault. However, in the previous cases mentioned those seem to be not submodules. One in torch/csrc/profiler/python/combined_traceback.cpp, and the other one in c10/util/StringUtil.cpp and torch/csrc/init_flatbuffer_module.cpp

[malfet]

@qqaatw links plz. In the meantime, testing how many compiler will raise only virtual members can be override error in #97782

[qqaatw]

@malfet here are the links:
#96878 (comment) #92755 (comment)