Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow user to change openai base url and model for LocalAI support #35

Merged
merged 2 commits into from
Sep 1, 2023

Conversation

barakplasma
Copy link
Contributor

by changing the openai base url, we can use LocalAI to summarize articles instead of ChatGPT.
this also lets the user change the model (gpt-4 or gpt-3.5-turbo or openllama)

also bumps the dependencies for:
github.com/sashabaranov/go-openai v1.14.2
github.com/spf13/viper v1.16.0

my go linter removed a bunch of trailing spaces by accident


I tested with and without the config for openai_base_url and openai_model to make sure it works for existing configs.

@barakplasma barakplasma changed the title feat: allow user to change openai base url for LocalAI support feat: allow user to change openai base url and model for LocalAI support Aug 28, 2023
@piqoni
Copy link
Owner

piqoni commented Sep 1, 2023

Very cool, this looks quite useful. I cannot truly test this (I do have llama2 on one machine but dont have LocalAI) but LGTM :) Thanks for the addition!

@piqoni piqoni merged commit 1a456de into piqoni:main Sep 1, 2023
1 check passed
@barakplasma barakplasma deleted the change-openai-base-url branch October 4, 2023 17:22
@barakplasma
Copy link
Contributor Author

barakplasma commented Oct 4, 2023

By the way, this works with litellm openai proxy and ollama + mistralai as well

@krrishdholakia
Copy link

Hey @barakplasma any tweaks required for litellm proxy besides the int(time.time) patch?

@barakplasma
Copy link
Contributor Author

barakplasma commented Oct 4, 2023

@krr

Hey @barakplasma any tweaks required for litellm proxy besides the int(time.time) patch?

I don't think any other changes are needed. In my local patch I wrapped all the other occurrences of time time with int, but doing it at the top of main py should be enough

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants