You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Expert Parallelism (MoE) feature we currently have cannot be used with data parallelism. we'll make it can be worked with Data Parallelism and reflects a new design that can further reduce the communication amount by 1.5 times.
Describe a TODO feature
Assignees
The text was updated successfully, but these errors were encountered: