Skip to content

Commit

Permalink
fix fused_qkv model accuracy issue (microsoft#5217)
Browse files Browse the repository at this point in the history
Fused_qkv model can not correctly choose the fused_qkv type. Need to
update the module_name_matches.

Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
  • Loading branch information
2 people authored and SNahir committed Mar 11, 2024
1 parent e2b7c2b commit cfcf665
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion deepspeed/module_inject/fusedqkv_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ def _transpose_fused_qkvw(src, mp_size, fused_qkv_type=None, module=None):

raise ValueError("unknown fused_qkv_type")

module_name_matches = [k for k in fused_type_dict.keys() if module_str in k]
module_name_matches = [k for k in fused_type_dict.keys() if k in module_str]
if module_name_matches:
# There can be overlap with matches (e.g., "DecoderLayer" and "FalconDecoderLayer").
# We take the longest matching module_name
Expand Down

0 comments on commit cfcf665

Please sign in to comment.