The English-Chinese translation is incomplete.

[detectRecog]

haoranxu/ALMA-7B-R
Downloading shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00, 1.92it/s]
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:28<00:00, 9.57s/it]
['Translate this from Chinese to English:\nChinese: 我爱机器翻译。\nEnglish: I love machine translation.']
['Translate this from English to Chinese:\nEnglish: The multi-modality large language model is designed for the big AGI industry.\nChinese：多模态大型语言模型是为大型人工智能行业设']
['Translate this from English to Chinese:\nEnglish: Chocolate Peanut Butter Protein Bars.\nChinese：巧克力芝士蛋白饼干。']

I write two random examples and want to convert the English sentences to Chinese:
The wired things happened that the output is always in-complete. (1-3 words are missing.)

The codes to reproduce are provided as follows:

name = "haoranxu/ALMA-7B-R"
print(name)
# Load base model and LoRA weights
model = AutoModelForCausalLM.from_pretrained(name, torch_dtype=torch.float16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(name, padding_side='left')

# Add the source sentence into the prompt template
prompts = ["Translate this from Chinese to English:\nChinese: 我爱机器翻译。\nEnglish:",
           "Translate this from English to Chinese:\nEnglish: The multi-modality large language model is designed for the big AGI industry.\nChinese：",
           "Translate this from English to Chinese:\nEnglish: Chocolate Peanut Butter Protein Bars.\nChinese：", # 巧克力花生酱蛋白棒
           ]

for prompt in prompts:
    input_ids = tokenizer(prompt, return_tensors="pt", padding=True, max_length=40, truncation=True).input_ids.cuda()
    # Translation
    with torch.no_grad():
        generated_ids = model.generate(input_ids=input_ids, num_beams=5, max_new_tokens=20, do_sample=True, temperature=0.6, top_p=0.9)
    outputs = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
    print(outputs)

[fe1ixxu]

Thanks for your interest!

Increasing the max_new_tokens from 20 to a larger number like 100 should fix issue of the incomplete translation.

[detectRecog]

The problem is solved. Thanks! @fe1ixxu