[BUG]: multi machines, multi cards training is stucked while "Launch model Parallel"

[shileims]

🐛 Describe the bug

Environment

pytorch==1.11, cuda==11.3, colossalai==0.3.0
transformers==4.28.1

[flybird11111]

May I take a look at your script commands?

🐛 Describe the bug

Environment

pytorch==1.11, cuda==11.3, colossalai==0.3.0 transformers==4.28.1

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

May I take a look at your script commands?

🐛 Describe the bug

Environment

pytorch==1.11, cuda==11.3, colossalai==0.3.0 transformers==4.28.1