Feature: Google Gemini and Local LLM Support

[sumant1122]

I see that OpenAI is being used for LLM tasks, but not everyone has access to OpenAI paid account. Google Gemini support would be great. A function to connect to local LLM would be great too.

[cyyeh]

@sumant1122 welcome to check out this issue: #277

[cyyeh]

Since we are currently working hard on improving end-to-end user experience. We may not at the moment have enough time to support other LLMs and do prompt engineering to other LLMs also. If you have thoughts on how to make the process easily scalable, we definitely welcome your feedback and contribution :)

For customizing your LLM or Document Store, please also refer to the guide: https://docs.getwren.ai/installation/custom_llm

Thank you

[sumant1122]

Thank you. Will check out the docs and try to contribute.

[cyyeh]

all, the ollama has been integrated in this branch, also you can use openai api compatible llm: chore/ai-service/update-env
we'll merge this branch to the main branch in the near future and update the documentation
as of now, I'll delete the original ollama branch
Thank you all for your patience

related pr: #376

[cyyeh]

All, we now support using Ollama and OpenAI API-compatible LLMs now with the latest release: https://github.com/Canner/WrenAI/releases/tag/0.6.0

Setups on how to run Wren AI using custom LLMs: https://docs.getwren.ai/installation/custom_llm#running-wren-ai-with-your-custom-llm-or-document-store

Currently, there is one obvious limitation for custom LLMs: you need to use the same provder(such as OpenAI, or Ollama) for LLM and embedding model. We'll fix that and release a new version soon. Stay tuned 🙂

I'll close this issue as completed now.