-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature Request: Integration of OpenAI Embeddings #101
Comments
Hi @argen666, welcome! Please let me know which ways are you thinking to integrate embeddings; |
Hi @enricoros, I guess the basic use case is to build a more complete research assistant trained on multiple custom documents. The basic step-by-step guide using embeddings:
In our case, I think we need to add support for the vector databases listed above and add configuration for connecting to them in the application settings. |
@argen666 @enricoros I have made a PR for this here, it is a decent start functionality-wise as a proof of concept I know it could be better integrated into the current codebase and have a better UI for sure |
@michaelcreatesstuff @enricoros Great work! I also implemented this functionality in parallel with you. I'm not creating a PR yet because I'm waiting for langchainJS to add the implementation to work with Redis and other vector databases. At the moment, I also have to use Pinecone because of these limitations. |
@argen666 thanks. Agreed, langchainJS seems a bit behind langchain python. I'm going to try python + FastAPI Have you tried this? It was on my list of concepts to explore https://js.langchain.com/docs/modules/indexes/vector_stores/integrations/memory |
@michaelcreatesstuff Thanks. I haven't tried that since I decided to focus on external vector stores to have an independent knowledge base |
@enricoros @michaelcreatesstuff Hi Team, I have made a pull request for this feature |
I believe Big-AGI could benefit greatly from embeddings as this could allow for exploration of new use cases and extended functionalities for the code assistant and textual contexts. Here is an attempt to provide a proper request description using the repo template to help continue the discussion. Why Description Requirements
(Generated with big-AGI using GPT4(1106) and vetted by the author of this post) |
Thanks for the description, clearly made by GPT-4 because it sounds good, but it's low on details. I read when to generate and where to store. But how are the embeddings being used? Just storing them is not enough. Is the objective to have a RAG use case? Embeddings can be used for many purposes, and I'd be curious about the top ways to use them. (Rag, MemGPT-like, etc.) |
I can share here my use cases here:
I hope this adds to the conversation. I would love to lend a hand to make this land on big-AGI. |
@bbaaxx its next on my list! Just need to get WSL working |
I would like to request the integration of OpenAI embeddings into the project. As OpenAI offers powerful language models, incorporating their embeddings could significantly improve the performance and capabilities of our project.
Please let me know if there are any concerns or additional requirements for implementing this feature. I am more than happy to contribute to the development and testing process.
The text was updated successfully, but these errors were encountered: