community: Add support for Upstash Vector

[ytkimirti]

This PR adds support for Upstash Vector database.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 20, 2024 10:50am

[baskaryan]

@ytkimirti feel free to tag when pr is ready for review!

[ytkimirti]

Hey @baskaryan, the code and integration tests are ready for review. If everything is ok I will start filling up the docs.

One question about dependencies, should I add upstash-vector as an optional dependency to /libs/community/pyproject.toml?
I was not sure because I couldn't find pinecone or chromadb there

[ytkimirti]

@baskaryan bump

[baskaryan]

code looks good to me! no need to add as optional dependency

if helpful, you can run local formatting and linting with:

cd libs/community
make format
make lint

[ytkimirti]

Thanks for the review, I will complete the docs and examples in a few days

About the new integration tests, they need these environment variables, you can create them from here if needed

UPSTASH_VECTOR_URL=your_upstash_vector_url
UPSTASH_VECTOR_TOKEN=your_upstash_vector_token

[ytkimirti]

Hello, @baskaryan! The pr is ready for merging. I've ran the integration tests locally and also tested the retriever API—everything seems to work.

[efriis]

Hey there! These integration tests will not work as-is in community. Can we:

• convert the integration tests into mocked unit tests

https://python.langchain.com/docs/contributing/testing#integration-tests

[ytkimirti]

Hey @efriis, do you mean mocking the upstash_vector library or just recording the requests using pytest-vcr? When writing the unit tests I looked at how other vectorstores wrote it (for example chroma) and I didn't see any mocking. Can you give more information about this?

[ytkimirti]

bump @efriis

[ytkimirti]

@baskaryan @efriis If it's not possible, we can remove the integration tests for now so we can merge this pr. What do you think about that?

[ytkimirti]

Update: We are currently adding support for metadata filtering. I will update this pr when the feature is shipped to the sdk

[efriis]

closing for now and can reopen when that's in!