Add settings for custom base url #2594
Conversation
Making the openai base url and embedding dimension configurable, these are useful to integrate AutoGPT with other models, like LLaMA
|
LGTM |
|
A name like |
|
It's still using the OpenAI API, just not their endpoint, even if the model behind it isn't an OpenAI model. |
|
LGTM 👍 |
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #2594 +/- ##
==========================================
- Coverage 49.65% 41.41% -8.24%
==========================================
Files 64 63 -1
Lines 3021 3011 -10
Branches 505 495 -10
==========================================
- Hits 1500 1247 -253
- Misses 1401 1698 +297
+ Partials 120 66 -54
... and 17 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
github-actions
bot
added
the
conflicts
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
|
Personal note: sorry it has been taking so long. We're swamped in a huge re-arch and a ton of PRs and we're only now figuring out a more scalable team workflow. Thanks for staying with us :) |
|
Deployment failed with the following error: |
@Pwuts just approved, if it's ok for you too it can be merged |
|
You changed AutoGPT's behaviour. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged. |
|
You changed AutoGPT's behaviour. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged. |
|
@merwanehamadi I wouldn't know how this PR would change behavior.. Can you figure out why it's giving this comment? |
|
@Pwuts if the PR changed the behaviour once, currently it will keep sending the behaviour label. So this PR changed the behaviour in the past, most likely
|
|
You changed AutoGPT's behaviour. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged. |
|
We're aware that this functionality was broken in #4803 and are working to fix it |
Making the openai base url and embedding dimension configurable, these are useful to integrate AutoGPT with other models, like LLaMA
Background
This makes AutoGPT capable of connecting to custom openai-like APIs like [keldenl](https://github.com/keldenl/gpt-llama.cpp), and use other models, like LLaMA and derivates.
see also #25 #567 #2158
Changes
Added OPENAI_API_BASE_URL
and EMBED_DIMto .env_template and loaded them in config.py, making sure OPENAI_API_BASE_URL would be ignored if USE_AZURE is True.Also, modified the files in autogpt/memory to use the value in EMBED_DIM instead of 1536 (wich is still the default)UPDATE: embed_dim settings isn't needed anymore
Documentation
I added an explanation of what those new configurations do in the .env_template file, following the comments on other configurations
Test Plan
Tested it by using gpt-llama.cpp on my machine, and setting OPENAI_API_BASE_URL to the API url in my .env file.
I used Vicuna 13B, so i also set EMBED_DIM to 5120For this test, i also set OPENAI_API_KEY to the model's path (it's an "hack" made by gpt-llama.cpp to get the model's path)
PR Quality Checklist