Skip to content

Commit

Permalink
fix no_repeat_ngram_size_logit_process
Browse files Browse the repository at this point in the history
  • Loading branch information
yuehuayingxueluo committed May 10, 2024
1 parent 2d94b8c commit b06efbb
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion colossalai/inference/logit_processors.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@ def no_repeat_ngram_size_logit_process(logits, ngram_size: int, batch_token_ids:
batch_size = len(batch_token_ids)

for batch_id in range(batch_size):
current_token_ids = batch_token_ids[batch_id]
current_len = current_token_ids.size(0)
if current_len + 1 < ngram_size:
continue

current_token_ids = batch_token_ids[batch_id]
token_ids_list = current_token_ids.tolist()

ngrams_dict = {}
Expand Down

0 comments on commit b06efbb

Please sign in to comment.