You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
inference engine (using the BaseContainer instead of BaseMoeContainer) tries to transpose weights of MoE layer with logic written for a single MLP layer
Expected behavior
Inference engine should load without an issue
The text was updated successfully, but these errors were encountered:
Describe the bug
Not able to start inference for MoE based DS Megatron GPT model because the MegatronMoELayerPolicy (https://github.com/microsoft/DeepSpeed/blob/master/deepspeed/module_inject/containers/megatron_gpt_moe.py) that supports this feature is not being used/referenced anywhere.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Inference engine should load without an issue
The text was updated successfully, but these errors were encountered: