Log generation only on rank 0 for distributed

[rohan-varma]

Changelog

• Avoids excessive log noise in the case of distributed.

Test plan

• Run finetune:

torchrun --nnodes 1 --nproc_per_node 8 recipes/finetune_llm.py --config recipes/configs/alpaca_llama2_finetune.yaml --autocast-precision bf16 --fsdp True --batch-size 1 --run-generation 50 --optimizer AdamW

[netlify]

✅ Deploy Preview for torchtune-preview ready!

Name	Link
🔨 Latest commit	ebe11a7
🔍 Latest deploy log	https://app.netlify.com/sites/torchtune-preview/deploys/659f1f39307d60000865fd92
😎 Deploy Preview	https://deploy-preview-171--torchtune-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[joecummings]

lgtm

[ebsmothers]

Personally I like having a utility for these things (e.g. print0 here). Curious if this falls under the category of things we don't want to abstract away in the recipe though. Either way, looks good!

[joecummings]

Personally I like having a utility for these things (e.g. print0 here). Curious if this falls under the category of things we don't want to abstract away in the recipe though. Either way, looks good!

I like this - especially b/c it might need to be repeated elsewhere.

[rohan-varma]

print0 makes a lot of sense, we can add that as a follow up: #172

[daniellepintz]

We can also have a util for this, e.g.:

https://github.com/pytorch/tnt/blob/725e06c3231b2fb895c86c5ccf30e08664072af7/torchtnt/utils/distributed.py#L106-L118