-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Open
Description
Hi, given I want to translate a simple sentence
python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "Translate me this sentence into french : the white rabbit jumps over the white rainbow" -n 100 -temp 0.1
but this keeps on repeating text or annotations, word per word translation until it reaches 100 tokens ... Is there any way of saying " Stop, once the appropriate answer is given ? "
Thanks a zillion time for any clue, notice, comment, enlightenment ;)
Metadata
Metadata
Assignees
Labels
No labels