Skip to content

Commit

Permalink
fix stablelm.py tensor-parallel-size bug (vllm-project#2482)
Browse files Browse the repository at this point in the history
  • Loading branch information
YingchaoX authored and hongxiayang committed Jan 18, 2024
1 parent 327ccab commit 3ee0259
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion vllm/model_executor/models/stablelm.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ def __init__(self,
self.q_size = self.num_heads * self.head_dim
self.kv_size = self.num_key_value_heads * self.head_dim

if (self.head_dim * self.num_heads) != self.hidden_size:
if (self.head_dim * self.num_heads * tp_size) != self.hidden_size:
raise ValueError(
f"hidden_size must be divisible by num_heads (got `hidden_size`: {self.hidden_size}"
f" and `num_heads`: {self.num_heads}).")
Expand Down

0 comments on commit 3ee0259

Please sign in to comment.