> Yeah, if someone could get a PR of a vision model working locally on the project that'd be great I think

[Andy1996247]

Yeah, if someone could get a PR of a vision model working locally on the project that'd be great I think

Would this work? https://llava-vl.github.io/

https://simonwillison.net/2023/Nov/29/llamafile/

Originally posted by @Andy1996247 in #86 (comment)

[BorisMolch]

From what i manually tested its not better than gptV...
i guess we have to wait for vision to improve and provide accurate coordinates #7

or take a different approach.

[joshbickett]

@BorisMolch even though Llava may not perform well, others may be interested to try it and see how they can improve it. If you want to make a PR for running Llava locally, I'd be happy to review it

[BorisMolch]

@BorisMolch even though Llava may not perform well, others may be interested to try it and see how they can improve it. If you want to make a PR for running Llava locally, I'd be happy to review it

"manually tested"
as in gave Llava screenshots and tried to see if its capable to instructing. its not (as well as GPTV)

[joshbickett]

oh ok, understood. Well thanks for the input nonetheless!

[joshbickett]

We now have Llava integrated thanks to the PR from @michaelhhogue!