Support memory profiling feature from on-demand path

[briancoutinho]

Summary

Currently the memory profiler feature in PyTorch is available via the profiler API by passing profile_memory=True in the interface. It is desirable to also enable memory profiling using the Kineto on-demand API.

Dynolog (profiling daemon) can communicate to a PyTorch program to enable on-demand profiling. Dynolog sends the configuration using a string. Dynolog already configuring the profile_memory option. However, the memory profiles are not obtained.

Understanding the code path.

Dynolog sends this string "PROFILE_PROFILE_MEMORY" as implemented here in PR159.

Inside Kineto

1. Profiler memory is accepted in Kineto config in Config.cpp
2. This is passed to the PyTorch client interface here
3. Pytorch client receives this and calls enableProfiler [here[(https://github.com/pytorch/pytorch/blob/660ec3d38d9d1c8567471ae7fe5b40ae7c6d7438/torch/csrc/profiler/kineto_client_interface.cpp#L44-L53C5)
4. We have this field being set in ProfilerConfig , details in observer.h
5. I lost track of how the config_ gets passed around from here. But i see it being checked in profiler_kineto.cpp

Few things to confirm

• Does it work with input shapes enabled
• dyno gputrace <..> --record_shapes --profile_memory