Add support for Llama2, Palm, Cohere, Replicate, HuggingFace (Starcoder) Models - using litellm

[ishaan-jaff]

This PR adds support for models from all the above mentioned providers using https://github.com/BerriAI/litellm/

Here's a sample of how it's used:

from litellm import completion, acompletion

## set ENV variables
# ENV variables can be set in .env file, too. Example in .env.example
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# llama2 call
model_name = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
response = completion(model_name, messages)

# cohere call
response = completion("command-nightly", messages)

# anthropic call
response = completion(model="claude-instant-1", messages=messages)

[ishaan-jaff]

Addresses #18

[ishaan-jaff]

@silvanmelchior if this initial commit looks good. - happy to add docs / testing too

[silvanmelchior]

I'm actually not sure how this could work; except for OpenAI no other model was finetuned to call functions (in this case the code interpreter). That's why for llama I had to do all this parsing of the code block.
Did you test this with code execution?

[silvanmelchior]

Close due to inactivity

[ishaan-jaff]

@silvanmelchior we can help with this issue: #29

[ishaan-jaff]

ahh I understand for non openai LLMs you'd need to extract just the executable code block ?

did I understand that correctly ?

[silvanmelchior]

Yes, or something smarter, if the model allows it

[ishaan-jaff]

sounds good - i'm happy to add StableCode #28 if you think that would be useful for IncognitoPilot

[silvanmelchior]

I would recommend to first hack together a prototype, so we see if the performance is useful enough. I think llama is already at the limit of what is usable, since ot was not finetuned for tool usage like the interpreter. (Even though I think llama is great!).

If such a prototype shows that it indeed is useful, I'm open to adding other models