Need help for multi nodes(machines) multi cards Ray + deepspeed z3 /fsdp

[wizounovziki]

My cases:
2 machines,16 GPUs (A800)

1. torch run sft lora: failed due to connection env config issue
2. Ray + sft: succeeded in native DDP &deepspeed z0.
   Seems that deepspeed z3 is not supported in Ray + Llama-factory, so I turned to fsdp and followed the steps from https://llamafactory.readthedocs.io/en/latest/advanced/distributed.html#id31, and take the config file for examples/accelerate/fsdp_config_multiple_nodes.yaml, but always get connection failed. I was thinking that since I set up ray and started with ray, the connection params should be ignored.
   Wondering how should I set these connection params?