Update ollama.py for image handling

[rick-github]

ollama wants plain base64 jpeg/png images, and some clients send dataURI and/or webp. Remove prefixes and convert all non-jpeg/png images to jpeg.

Fixes #1427, #2661

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 28, 2024 3:00pm

[rick-github]

Hi Jakob, base64 encoded data shouldn't contain commas, it's restricted to the character set [A-Za-z0-9/+=]: https://en.wikipedia.org/wiki/Base64

…

On Thu, 11 Apr 2024 at 05:10, Jakob ***@***.***> wrote: I think this implementation is flawed since you're doing image.split(",") for the comma you're expecting at the beginning, but the base64 encoded data itself can also contain commas. — Reply to this email directly, view it on GitHub <#2888 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADSBEJVKJCF77L263ZUULTTY4X5LTAVCNFSM6AAAAABF3HOAX6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBYHA2TKNBRGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[jakobdylanc]

Yeah realized that afterwards and deleted my comment :) sorry about that. I was confusing "," with "/"

[krrishdholakia]

@rick-github thanks for this - can you add a test with a mock call for this to test_completions.py we've had a couple regressions with merged ollama pr's recently.

place to add test -

litellm/litellm/tests/test_completion.py

Line 4 in b29994a

Example of a mock call -

litellm/litellm/tests/test_completion.py

Line 1314 in b29994a

def test_hf_test_completion_tgi():

[rick-github]

Done.

[krrishdholakia]

Thanks @rick-github just share a screenshot of it passing testing for your test + tests in test_ollama.py and we should be good to merge!

Thank you again for your awesome work here

[rick-github]

[rick-github]

[krrishdholakia]

LGTM!