Clarifai-LiteLLM : Added clarifai as LLM Provider.

[mogith-pn]

Objectives

• Added Clarifai as LLM provider.
• This enables to call LLM models hosted in clarifai platform.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 3, 2024 2:04pm

[krrishdholakia]

Thanks for this PR @mogith-pn. Happy to add support.

Blockers

• async completion
• async streaming
• streaming

Once completed, if you can share a screenshot of this passing testing for you - that would be great!

[mogith-pn]

Thanks for this PR @mogith-pn. Happy to add support.

Blockers

• async completion
• async streaming
• streaming

Once completed, if you can share a screenshot of this passing testing for you - that would be great!

Hi @krrishdholakia ,
Thanks for your response. Currently we don't support the streaming and async completion. So should I include a acompletion, stream function and raise NotImplemented error sort of ? or what do you suggest ?

[krrishdholakia]

For async completions - it's just a call with our async http handler

see anthropic -

litellm/litellm/llms/anthropic.py

Line 258 in 3d92876

self.async_handler = AsyncHTTPHandler(

For streaming - if your backend server doesn't support streaming, then make a normal completion/async completion call and wrap it in an iterator -

litellm/litellm/llms/anthropic.py

Line 211 in 3d92876

completion_stream = ModelResponseIterator(

This will make sure people's calls don't break in prod.

[krrishdholakia]

I don't think your PR uses the clarifai sdk. If i missed it - can you please use the HTTP endpoints and our httpx clients instead - e.g.

litellm/litellm/llms/anthropic.py

Line 258 in 3d92876

self.async_handler = AsyncHTTPHandler(

.

This will keep the package light, and let people switch between providers easily.

[mogith-pn]

Hi @krrishdholakia ,
Added the above changes.
Created a test case for async completion in test_clarifai_completion.py and for streaming added it in test_streaming.py. Both passed!.

[mogith-pn]

Hey @krrishdholakia ,
Hope you are doing good. Kindly take a look at it and let me know if this looks good..!

[krrishdholakia]

Great! planning on merging after we have a stable release out later today 🚀
thanks for the great work @mogith-pn

Curious - how're you using litellm today?

[mogith-pn]

Great! planning on merging after we have a stable release out later today 🚀 thanks for the great work @mogith-pn

Curious - how're you using litellm today?
@krrishdholakia ,
Thanks for the merge. Sorry I missed your question !
We were using it for prototyping and we used direct open AI models. but this was part of our integration goals to integrate clarifai as LLM providers with LiteLLM so our community opensource users can leverage it.