partners/astradb: bootstrapping Astra DB as Partner Package #16875

hemidactylus · 2024-02-01T12:43:06Z

Description: This PR introduces a new "Astra DB" Partner Package.

So far only the vector store class is duplicated there, all others following once this is validated and established.

Along with the move to separate package, incidentally, the class name will change AstraDB => AstraDBVectorStore.

The strategy has been to duplicate the module (with prospected removal from community at LangChain 0.2). Until then, the code will be kept in sync with minimal, known differences (there is a makefile target to automate drift control. Out of convenience with this check, the community package has a class AstraDBVectorStore aliased to AstraDB at the end of the module).

With this PR several bugfixes and improvement come to the vector store, as well as a reshuffling of the doc pages/notebooks (Astra and Cassandra) to align with the move to a separate package.

Dependencies: A brand new pyproject.toml in the new package, no changes otherwise.

Twitter handle: @rsprrs

…ake deletions optional

… dir, lint tests

…stra

…methods

…n packages

vercel · 2024-02-01T12:43:10Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 15, 2024 11:29pm

docs/docs/integrations/vectorstores/astradb.ipynb

cbornet

LGTM, great work !

Co-authored-by: Christophe Bornet <cbornet@hotmail.com>

cbornet · 2024-02-11T13:30:23Z

It would be convenient to have an astradb package extra that would pull langchain-astradb. But that can be done in future work.

efriis

Some comments!

efriis · 2024-02-07T17:58:05Z

libs/community/langchain_community/vectorstores/astradb.py

instead of these changes in community, let's just leave it as-is and add a @deprecated(..., alternative_import="langchain_astradb.AstraDBVectorStore"). People can update to the new one that way!

@efriis do you mean that we don't try to keep community and partner code in sync ? And won't do any evolution on community side ?

@efriis So we can work on the assumption that the merged PR and the new package on PyPI will essentially happen at once?
I am asking because, if so, I can go ahead and update most imports in the example notebooks and such (which was originally kept as a 2nd-phase step, after package published)

Feel free to still do as second step. Easier to validate that everything is working as intended

efriis · 2024-02-13T20:05:21Z

libs/partners/astradb/langchain_astradb/vectorstores/astradb.py

+        Create an AstraDBVectorStore vector store object. See class docstring for help.
+        """
+        try:
+            from astrapy.db import (


all imports can now be top-level as a concrete dep! No more optional needed, and can properly lint too :)

efriis · 2024-02-13T20:05:36Z

libs/partners/astradb/pyproject.toml

+
+[tool.poetry.dependencies]
+python = ">=3.8.1,<4.0"
+langchain-core = ">=0.0.12"


Change to ^0.1

efriis · 2024-02-13T20:06:25Z

libs/partners/astradb/tests/integration_tests/vectorstores/test_astradb.py

+        export ASTRA_DB_API_ENDPOINT="https://<DB-ID>-us-east1.apps.astra.datastax.com"
+        export ASTRA_DB_APPLICATION_TOKEN="AstraCS:........."


add necessary secrets to .github/workflows/_release.yml and .github/workflows/_integration_test.yml

…emoved sync check from makefile, adding deprecation to community module

…normalize Makefile

hemidactylus · 2024-02-15T23:06:27Z

Hello, @efriis , I think I have addressed all your remarks. I am going to share the CI/CD secrets in a direct message momentarily.

I'd appreciate a quick confirmation that the PR is now as intended :) thank you!

cbornet · 2024-02-16T01:53:27Z

Awesome 😃 ! Can't wait to see the first package on Pypi !

**Description:** This PR introduces a new "Astra DB" Partner Package. So far only the vector store class is _duplicated_ there, all others following once this is validated and established. Along with the move to separate package, incidentally, the class name will change `AstraDB` => `AstraDBVectorStore`. The strategy has been to duplicate the module (with prospected removal from community at LangChain 0.2). Until then, the code will be kept in sync with minimal, known differences (there is a makefile target to automate drift control. Out of convenience with this check, the community package has a class `AstraDBVectorStore` aliased to `AstraDB` at the end of the module). With this PR several bugfixes and improvement come to the vector store, as well as a reshuffling of the doc pages/notebooks (Astra and Cassandra) to align with the move to a separate package. **Dependencies:** A brand new pyproject.toml in the new package, no changes otherwise. **Twitter handle:** `@rsprrs` --------- Co-authored-by: Christophe Bornet <cbornet@hotmail.com> Co-authored-by: Erick Friis <erick@langchain.dev>

hemidactylus added 28 commits January 16, 2024 18:10

moved the vector store

ff6a9a1

reinstate astradb vector store in community

a8a0d26

made modules identical across copies

4bbdfab

fix Document import

bac9099

move imports

c90120a

adapt pre_delete_collection. Testing: optimize collection usage and m…

a22defa

…ake deletions optional

mmr utils in package, decoupling from community

21222e2

AstraDB vector store as a deprecated alias

b365f84

refactor astradb test cred fixture

3a445fd

make target for transitional code sync; typed credentials dict

6de922b

Merge branch 'master' into SL-astradb-as-partner

aa1f4f8

merge latest master

23af809

bump astrapy, add note on mmr duplicated utility, remove package docs…

b0c8e96

… dir, lint tests

align vectorstore across duplicates

f244748

refactor vectorstores and test to be more structured

664541d

main provider page and vectorstore notebook split in two: cassandra/a…

0a77910

…stra

pruned cassandra->astra redirects in vercel.json

7b9b57c

a minimal readme

4c3a25a

WIP merging async AstraDB vector store

c4bbc18

wip2 mypy checks

74f6e47

clean mypy on community astradb vector store

80f3bcd

async in docstring, small fixes to the test

235d9aa

cassandra attribution statement in docs/notebooks; add note on async …

0bf1d18

…methods

fix never-awaited warning on _setup_db; align vectorstore code betwee…

6bcedf2

…n packages

vendor-agnostic cassandra.mdx

5437ec0

uniform naming + alias in community

34dbe54

bring latest master in

242273c

dependency upgrade in notebook

a0220f9

efriis added the partner label Feb 1, 2024

efriis self-assigned this Feb 1, 2024

dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Feb 1, 2024

dosubot bot added Ɑ: vector store Related to vector store module 🤖:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features labels Feb 1, 2024

vercel bot deployed to Preview February 1, 2024 12:50 View deployment

protect astrapy import in community integration test

35b3f7c

vercel bot deployed to Preview February 1, 2024 13:37 View deployment

cbornet reviewed Feb 1, 2024

View reviewed changes

docs/docs/integrations/vectorstores/astradb.ipynb Outdated Show resolved Hide resolved

cbornet approved these changes Feb 1, 2024

View reviewed changes

Update docs/docs/integrations/vectorstores/astradb.ipynb

48bf262

Co-authored-by: Christophe Bornet <cbornet@hotmail.com>

vercel bot deployed to Preview February 1, 2024 14:56 View deployment

cbornet mentioned this pull request Feb 3, 2024

Use async embedding when provisionning the AstraDB VectorStore if in the event loop #16841

Closed

bring in the type:ignore specializations from master

b859ccb

vercel bot deployed to Preview February 6, 2024 09:30 View deployment

efriis reviewed Feb 13, 2024

View reviewed changes

hemidactylus added 5 commits February 15, 2024 12:13

latest master

d2e0539

langchain-core dependency version upgrade

12d7d5b

Astra DB integration testing secrets into gh workflow yamls

731cb5e

brought community/partner Astra DB vector stores out of sync again. R…

4ce7703

…emoved sync check from makefile, adding deprecation to community module

astra db p.p. vectorstore: remove try/imports, bump astrapy version, …

cf58e52

…normalize Makefile

vercel bot deployed to Preview February 15, 2024 23:10 View deployment

efriis added 2 commits February 15, 2024 15:19

remove optional secret

6a6c063

Merge branch 'master' into SL-astradb-as-partner

5b1a417

vercel bot deployed to Preview February 15, 2024 23:29 View deployment

efriis merged commit 5240eca into langchain-ai:master Feb 15, 2024
94 checks passed

hemidactylus deleted the SL-astradb-as-partner branch February 16, 2024 09:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

partners/astradb: bootstrapping Astra DB as Partner Package #16875

partners/astradb: bootstrapping Astra DB as Partner Package #16875

hemidactylus commented Feb 1, 2024

vercel bot commented Feb 1, 2024 •

edited

cbornet left a comment

cbornet commented Feb 11, 2024

efriis left a comment

efriis Feb 7, 2024

cbornet Feb 14, 2024

hemidactylus Feb 15, 2024

efriis Feb 15, 2024

efriis Feb 13, 2024

efriis Feb 13, 2024

efriis Feb 13, 2024

hemidactylus commented Feb 15, 2024

cbornet commented Feb 16, 2024

		export ASTRA_DB_API_ENDPOINT="https://<DB-ID>-us-east1.apps.astra.datastax.com"
		export ASTRA_DB_APPLICATION_TOKEN="AstraCS:........."

partners/astradb: bootstrapping Astra DB as Partner Package #16875

partners/astradb: bootstrapping Astra DB as Partner Package #16875

Conversation

hemidactylus commented Feb 1, 2024

vercel bot commented Feb 1, 2024 • edited

cbornet left a comment

Choose a reason for hiding this comment

cbornet commented Feb 11, 2024

efriis left a comment

Choose a reason for hiding this comment

efriis Feb 7, 2024

Choose a reason for hiding this comment

cbornet Feb 14, 2024

Choose a reason for hiding this comment

hemidactylus Feb 15, 2024

Choose a reason for hiding this comment

efriis Feb 15, 2024

Choose a reason for hiding this comment

efriis Feb 13, 2024

Choose a reason for hiding this comment

efriis Feb 13, 2024

Choose a reason for hiding this comment

efriis Feb 13, 2024

Choose a reason for hiding this comment

hemidactylus commented Feb 15, 2024

cbornet commented Feb 16, 2024

vercel bot commented Feb 1, 2024 •

edited