Add USE_C10D_NCCL around NCCL trace utils

[kwen2501]

Fixes #114575

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @awgu @penguinwu @fegin @XilunWu @wanchaol @fduwjj @wz337 @tianyu-l @wconstab @yf225 @kiukchung @d4l3k @LucasLLC

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/114597

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 3eb9cc0 with merge base 624f202 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[malfet]

LGTM, but can we add a test of sorts (is there distributed C++ tests, that just build?)

Please note, that this flag would be set to false for extensions, which probably defeats the purpose.
Please consider dependening on the macros defined in say https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/cuda/CUDAConfig.h.in or in https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/Config.h.in (former is preferable imo)

[kwen2501]

Thanks for the pointers @malfet !
The DebugInfoWriter and NCCLTraceBuffer were recently added to for use by ProcessGroupNCCL only. And it was moved out to TraceUtil.h for code cleanness of ProcessGroupNCCL.cpp. (TraceUtil.h was primarily used by ProcessGroupNCCL only as well.) Indeed there would be some more work if we were to generalize the infra in TraceUtil.h for other backends' use.

[kwen2501]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here