llava-cli outputs gibberish #6944
Description
I am using an M1, on commit 928e0b7.
When I run
./llava-cli -m ./models/llava-v1.6-mistral-7b/ggml-mistral-7b-q_5_k.gguf --mmproj ./models/llava-v1.6-mistral-7b/mmproj-mistral7b-f16-q6_k.gguf -p 'Describe the image.' --image ./models/llava-v1.6-mistral-7b/S.jpg -c 4096
I get:
to to a,, new do s is and d t, in in is, r is to and,, h to a and is is t.. for s has m d,., a. is,, m he un, and b st a to to
#. r. d t n is the to.' with, l the to., is is and t r and a t in, re h s d a is, is l in in as as is r in un,., to t t h a.. t on as a, st' > > the to a, t r is a the, d # is and the p r as t is, as, h has to do the in. in as c m. a is l the is to n st has on on r t, s' is new to, t a is, and/ he in g and/ is, is re, with at c is in' d,. at p c and/ is is.. n is h on a the and/,. is,- b the the on. at is is h do. and/ re, r in s, on for to to p b n,.. a as
I also tried with xtuner/llava-phi-3-mini with similar results.
./llava-cli -m ./models/llava-phi-3-mini/ggml-model-f16.gguf --mmproj ./models/llava-phi-3-mini/mmproj-model-f16.gguf -p 'Describe the image.' --image ./models/llava-v1.6-mistral-7b/S.jpg -c 4096
â }
on }RESS, and thus,uminate, whereas, }}raint, but, }, }ioned, while, }}RESS, especially and hence, }ktion, }RESS,RESS,abeth,uminate,オ,RESS, }ività,RESS,ício,raint, }derr, }derr, and thus,RESS, }ución, }abeth, }}rache,RESS,abeth, }RESS, }RESS, }ionali,RESS, }}RESS, }eign, and thus,RESS, }abeth,RESS,ício,ioned,abeth,ício,derr,iry,RESS, }RESS, }RESS,otal,umm,RESS,RESS,オ, }RESS,annels, }iseconds, }}}}abeth,bose,abeth,ershell, }ktion, }RESS, }uminate,RESS, }idor, }RESS,ktion,abeth,ício,RESS,オ, }esses, perhapsRESS,abeth, }}abeth,RESS,RESS,RESS,uminate,derr,derr,bose,ività, makunivers, }}ício, }ioned,otal,ershell,abeth,RESS,RESS,RESS, }}`RESS,RESS,RESS,RESS,abeth,uminate,
Have there been any breaking changes to the llava-cli command? Can someone reproduce this issue as well?