feat: add MemorystoreDocumentSaver class #18
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
product-auto-label
bot
added
the
api: redis
craig08
force-pushed
the
doc-saver
branch
2 times, most recently
from
3dfaa0b to
b6c03e3
Compare
PingXie
reviewed
Feb 13, 2024
craig08
force-pushed
the
doc-saver
branch
2 times, most recently
from
3a89de6 to
e1dfe23
Compare
PingXie
reviewed
Feb 14, 2024
craig08
added a commit
that referenced
this pull request
Feb 16, 2024
* feat(vector_store): Implement RedisVectorStore class This commit introduces the RedisVectorStore class, extending VectorStore to enable vector storage and retrieval using Memorystore Redis. * feat: add integration test for MemorystoreChatMessageHistory (#13) * fix: fix linter and encoding of MemorystoreChatMessageHistory class. (#20) * fix: fix linter and encoding of MemorystoreChatMessageHistory class. * fix: use relative path for __init__ imports * feat(vector store): Add usage demo and enhance Redis integration (#17) * feat(vector store): Add usage demo and enhance Redis integration - Add Jupyter notebook in `docs/` for using langchain vector store with Memorystore Redis. - Update `vector_store.py` for improved performance and reliability with Redis. - Modify `__init__.py` to reflect package structure changes and new functionalities. - Include `requirements.txt` and `setup.py` for easy installation and dependency management. - Add `state_of_the_union.txt` in `docs/` as a sample dataset for the notebook demo. * incorporate review feedback * added missing license header * excluded the test file from license check * reformatted source files * incorporated review feedback * fixed lint errors * fixed more lint errors * removed key_prefix argument from vectorstore * incorporated review feedback * fixed formatting errors * fixed a bad merge * suppress mypy errors for setuptools * trying mypy.ini * remove setup.py and requirements * add numpy dependency * Update integration.cloudbuild.yaml * feat: add MemorystoreDocumentSaver class (#18) * feat: add MemorystoreDocumentSaver class * fix: use variable in the worker pool of the Cloud Build integration * fix: update doc saver for json encoding * fix: align key_prefix behavior * fix: add back key_prefix if keys_or_ids is given * fix: fix indent of the comments * fix: get encoder directly from the Redis client * fix: add pipeline support for Redis client * fix: reorder prefix and content_field in constructor * fix: insert prefix to all given document IDs * fix: fix tests prefix and chat encoding * feat: add MemorystoreDocumentLoader class (#24) * refactor(vector-store): enhance validation and initialization processes (#22) * feat(vector store): Add usage demo and enhance Redis integration - Add Jupyter notebook in `docs/` for using langchain vector store with Memorystore Redis. - Update `vector_store.py` for improved performance and reliability with Redis. - Modify `__init__.py` to reflect package structure changes and new functionalities. - Include `requirements.txt` and `setup.py` for easy installation and dependency management. - Add `state_of_the_union.txt` in `docs/` as a sample dataset for the notebook demo. * incorporate review feedback * added missing license header * excluded the test file from license check * reformatted source files * incorporated review feedback * fixed lint errors * fixed more lint errors * removed key_prefix argument from vectorstore * incorporated review feedback * fixed formatting errors * fixed a bad merge * suppress mypy errors for setuptools * trying mypy.ini * remove setup.py and requirements * add numpy dependency * refactor(vector-store): enhance validation and initialization processes - Add validation for unsupported data types and negative vector sizes in VectorIndexConfig. - Refactor field_name parameters in HNSWConfig and FLATConfig to support type hinting with optional strings. - Enhance RedisVectorStore initialization by enforcing type checks for client, index_name, and embedding_service. - Introduce an optional 'ids' parameter in methods to allow explicit document identifiers, improving document management. - Adjust method documentation and error messages for clarity and consistency. * Incorporated review feedback * fixed formatting * incorporated review feedback * formatting * fixed bugs * fix formatting * added vector store unittests * actualy added vector store tests * fix formatting * fix: fix styles and formats --------- Co-authored-by: Craig Chi <craigchi@google.com> * fix: rename doc_ to document_ for consistency * fix: add delete API for document saver and raise error if key exists * fix: batch delete keys in document saver if specified --------- Co-authored-by: Ping Xie <pingxie@google.com> Co-authored-by: Ping Xie <pingxie@outlook.com> Co-authored-by: Averi Kitsch <akitsch@google.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #14 🦕