Enable support for custom openAI-compatible models (running code llama)

[elabbarw]

Describe the need of your request

Code Llama is live and kicking ass on Visual Studio Code via continue.dev. Some catching up is needed!

Proposed solution

Add the option for the following:

• custom API Base
• custom API key
• custom model name (e.g: CodeLlama-7b-Instruct-hf)

Additional context

Ref: https://github.com/lm-sys/FastChat/blob/main/docs/openai_api.md

[geekidentity]

Are there any plans to support llama?

[Gnzlt]

It would be great to support simple localhost APIs, like the one from @jmorganca Ollama project.
https://github.com/jmorganca/ollama/blob/main/docs/api.md

[siilats]

We have some You.com sponsor money so $100 and a badge of thanks if someone would like to propose a pull request on this! If you get a local LLAMA working, I can offer a machine that can run the 70bn parameter model with llama.cpp

[LynxPDA]

I already successfully use llama-cpp-python[server] with CodeGPT because llama-cpp-python[server] has an OpenAI-compatible API.

https://github.com/abetlen/llama-cpp-python

However, when processing promt for a long time, there can be problems due to timeout. It would be nice to set CodeGPT timeout separately.