Unify database / graph / vector-store node naming and classType #1635
Replies: 3 comments
🤖 Internal: Discord sync markerAuto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete. |
|
One caution on the migration that's easy to miss: a rename here is really two changes, and only one of them is dangerous. The folder/class name is cosmetic. The thing saved pipes actually key off is the
So there's a real fork in the road per node:
My vote is the full version for anything we rename, precisely so we don't leave a half-renamed layer behind — but if we want to stage it, the name-only step is a safe first move that breaks nothing, and we can follow with the provider + upgrade later. Just worth being explicit that the provider change is the one that carries migration risk, not the rename itself. |
|
To keep this reviewable, I'd split it into staged PRs where the risky part (the migration) lands last and on its own. Rough shape: PR 1 — renames only, no provider change. Just the PR 2 — extract PR 3 — migrate the rest. Weaviate, Milvus, Pinecone, Chroma, Astra, Atlas, Elasticsearch onto the base from PR 2. Bigger PR but mechanical and repetitive — every node looks like the qdrant one, so it reviews fast even though it's wide. PR 4 — provider renames + migration. Only now touch So the whole thing is safe and non-breaking right up until PR 4, and PR 4 is the only one a reviewer has to look at with migration in mind. Two side notes:
Does this order work for everyone? If so I'll open PR 1 (renames only) to get the easy part out of the way. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
We've been talking about cleaning up node naming, so I went digging through the nodes to figure out what a "consistent pattern" would actually look like. It really comes down to one family that never got the treatment the others did. Writing it up here so we can agree on the shape before anyone starts renaming things.
Where we already have a pattern
Two families are already consistent: a shared base class under
packages/ai/src/ai/common/<family>/, a matchingclassType, and a<family>_prefix.db_*→ relational,DatabaseGlobalBase/DatabaseInstanceBase, natural language → SQL.graph_*→GraphGlobalBase/GraphInstanceBase(extracted in feat(nodes, ai): add graph base class; make FalkorDB a real graph node #1584, applied to Neo4j in feat(nodes): migrate neo4j onto the graph base class (graph_neo4j) #1611), NL → Cypher.The vector databases never got this. They have no prefix,
classType: ["store"], and no shared base — each one reimplements lifecycle/config/IO straight on top of the genericIGlobalTransform. That's the real inconsistency, and it's why the tree is hard to navigate.storeanddatabaseare different capabilitiesThe names are close enough that it's worth being explicit: they do different jobs.
storeis a vector database (embeddings + semantic search),databaseis relational/SQL. Postgres is the clearest example, since it shows up as two separate nodes, one for each:db_postgresvectordb_postgresclassType["database"]["store"]get_data/execute)DatabaseBaseIGlobalTransformSame engine, two nodes, because they do two different things. Qdrant, Weaviate, Milvus, Pinecone, Chroma, Astra and Atlas are all
storeas well — their own descriptions say "A vector database component."(Heads up while we're here: Supabase today is a variant of
db_postgres, the relational one, viadb_postgres/services.supabase.json.)classTypeis a list of capabilities, not a single prefixAn engine can do more than one thing, so
classTypeshould carry every capability, and the name prefix should follow the primary one:["database", "store"]["graph", "database"]["graph", "database"]["store"]It's also why I'd push back on renaming
atlas→db_atlas: Atlas is a vector db (store), not a relationaldatabase. If prefix follows family it should bevectordb_atlas, notdb_. Otherwise we just trade one inconsistency for another.What I'm proposing
Prefix = primary capability,
classType= the full list.classTypeclassTypedb_postgres[database]db_postgres[database, store]DatabaseBase+VectorStoreBase*store; absorbvectordb_postgresvectordb_postgres[store]db_postgres)db_postgres(supabase)[database]db_supabase[database]DatabaseBasedb_arango[database]graph_arango[graph, database]GraphBase+DatabaseBasedb_hydradb[database]graph_hydradb[graph, database]GraphBase+DatabaseBasedb_mysql[database]db_mysql[database]DatabaseBasedb_clickhouse[database]db_clickhouse[database]DatabaseBasegraph_falkordb[graph]graph_falkordb[graph]GraphBasegraph_neo4j[graph]graph_neo4j[graph]GraphBaseqdrant[store]vectordb_qdrant[store]VectorStoreBase*weaviate[store]vectordb_weaviate[store]VectorStoreBase*milvus[store]vectordb_milvus[store]VectorStoreBase*pinecone[store]vectordb_pinecone[store]VectorStoreBase*chroma[store]vectordb_chroma[store]VectorStoreBase*astra_db[store]vectordb_astra[store]VectorStoreBase*atlas[store]vectordb_atlas[store]VectorStoreBase*db_atlas)index_search[store]vectordb_elasticsearch[store]VectorStoreBase**VectorStoreBasedoesn't exist yet. We'd extract it first, the same wayGraphBasecame out in #1584, before touching any of the vector nodes.Rollout
upgradeComponent— same migration we did for the graph renames in feat(nodes): migrate neo4j onto the graph base class (graph_neo4j) #1611.VectorStoreBasebehind the existing nodes, (2) migrate the pure vector nodes onto it with rename + upgrade, (3) unify the multi-capability engines (Postgres, Arango, HydraDB) last.What I want to agree on
classTypecarries the full list — good?vectordb_(matches the existingvectordb_postgres) orstore_?[database, store], or keep the relational and vector nodes split? (Supabase stays relational-only — that node isn't intended as a vector db.)If we're roughly aligned I'll start with extracting
VectorStoreBaseso the rest can hang off it.All reactions