refactor: createVectorStoreNode refactoring and embeddings batching support (no-changelog)

[OlegIvaniv]

Summary

• Refactor vector store node architecture into modular operation-based components
• Implement batched document embedding to reduce embeddings API calls and speed-up insertion
• UI rendering performance optimizations with debounced node updates

Changes

Vector Store Architecture Improvements

• Restructured createVectorStoreNode into separate modules organized by operation type
• Created dedicated handlers for each operation (load, insert, retrieve, update, retrieve-as-tool)
• Added test suite for each operation handler

Embedding Performance Optimization

• Implemented batched document processing for vector store insertions
• Added configurable batch size parameter to control embedding operations
• For node version 1.1+, documents are processed in batches instead of individually
• Single embedding operation performed per batch to significantly reduce API calls

UI Rendering Performance

• Added debounced rendering of workflow nodes and connections in canvas
• Implemented dynamic debounce timing based on workflow execution state:
  • Fast updates during editing (0ms debounce)
  • Throttled updates during execution (200ms debounce, 50ms max wait)
• Reduces CPU/memory usage during execution with large workflows

In my tests populating 500 items to QDrant vector store went from ~5mins to 10s.

Before

CleanShot.2025-03-04.at.12.50.00121221.mp4

After

CleanShot.2025-03-04.at.12.46.28.mp4

Related Linear tickets, Github issues, and Community forum posts

https://linear.app/n8n/issue/AI-749

Review / Merge checklist

• PR title and summary are descriptive. (conventions)
• Docs updated or follow-up ticket created.
• Tests included.
• PR Labeled with release/backport (if the PR is an urgent fix that needs to be backported)

[codecov]

Codecov Report

Attention: Patch coverage is 82.65306% with 34 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...red/createVectorStoreNode/createVectorStoreNode.ts	45.45%	18 Missing ⚠️
...re/VectorStorePinecone/VectorStorePinecone.node.ts	0.00%	2 Missing ⚠️
..._store/VectorStoreQdrant/VectorStoreQdrant.node.ts	0.00%	2 Missing ⚠️
...re/VectorStoreSupabase/VectorStoreSupabase.node.ts	0.00%	2 Missing ⚠️
...reateVectorStoreNode/operations/updateOperation.ts	91.66%	2 Missing ⚠️
...editor-ui/src/components/canvas/WorkflowCanvas.vue	94.59%	2 Missing ⚠️
...re/VectorStoreInMemory/VectorStoreInMemory.node.ts	0.00%	1 Missing ⚠️
...re/VectorStorePGVector/VectorStorePGVector.node.ts	0.00%	1 Missing ⚠️
...rePineconeInsert/VectorStorePineconeInsert.node.ts	0.00%	1 Missing ⚠️
...rStorePineconeLoad/VectorStorePineconeLoad.node.ts	0.00%	1 Missing ⚠️
... and 2 more

📢 Thoughts on this report? Let us know!