Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for text embeddings #36

Closed
4 tasks done
vasinov opened this issue May 15, 2023 · 0 comments
Closed
4 tasks done

Add support for text embeddings #36

vasinov opened this issue May 15, 2023 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@vasinov
Copy link
Member

vasinov commented May 15, 2023

We need a Griptape-native way to work with text embeddings and vector databases in order to support the following requested functionality:

  • Enable LLMs to access processed text data in vector databases via Griptape tools.
  • Help users generate text embeddings for their existing text and use it in Griptape tools.

Here is a general outline of proposed embeddings components and improvements

  • Generic BaseEmbeddingDriver and OpenAiEmbeddingDriver.
  • Update TextDocument to store and lazily generate embeddings.
  • Generic BaseVectorStorageDriver abstraction to provide a generic way to load and store embeddings.
  • PineconeVectorStorageDriver to be the first vector driver in order to support the most popular vector DB.
@vasinov vasinov added the enhancement New feature or request label May 15, 2023
@vasinov vasinov self-assigned this May 15, 2023
@vasinov vasinov closed this as completed May 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant