[Ray Train] Explain how to set timeout when using PyTorch Lightning Trainer

[scottsun94]

Description

It seems that users need to set timeout using torchConfig.

It's not clear when I should set it in ray.train.lightning.LightningConfigBuilder.strategy or in TorchCnfig.

Link

No response

[scottsun94]

cc: @woshiyyya @matthewdeng

[woshiyyya]

Yeah that's a problem, and I think this one of the action items for unifying configurations between lightning and AIR.

Currently we have two sets of configuration for Lightning and Ray AIR (checkpoint configs, backend configs, and scaling configs), which makes it hard for users to figure out the right place to provide them.

The ideal state from my mind is the user only need to provide lightning config, and we'll create a corresponding AIR config for them. But still there are many details to consider. I can draft a proposal on this topic later.

[stale]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.

[scottsun94]

This should be straightforward in 2.7? @woshiyyya

[woshiyyya]

Actually not. For the new API, they still need to specify timeouts in TorchConfig instead of RayDDPStrategy.

I think we still need to update the docstring for RayDDPStrategy to clarify this.