Skip to content

Ratnesh-181998/vector-database

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
Embeddings & Vector Stores.pdf
Embeddings & Vector Stores.pdf
 
 
Generative Al Design Patterns.pdf
Generative Al Design Patterns.pdf
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
Vector Database Architecture Cheat Sheet.pdf
Vector Database Architecture Cheat Sheet.pdf
 
 
Vector Databases Introduction by Nitin Borwankar.pdf
Vector Databases Introduction by Nitin Borwankar.pdf
 
 
What is Text Chunking Process.pdf
What is Text Chunking Process.pdf
 
 
𝐂𝐡𝐮𝐧𝐤𝐢𝐧𝐠 𝐢𝐬 𝐬𝐢𝐥𝐞𝐧t 𝐛𝐮𝐭 𝐢𝐭 𝐝𝐞𝐜𝐢𝐝𝐞𝐬 𝐲𝐨𝐮𝐫 𝐀𝐈’𝐬 𝐢𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞.pdf
𝐂𝐡𝐮𝐧𝐤𝐢𝐧𝐠 𝐢𝐬 𝐬𝐢𝐥𝐞𝐧t 𝐛𝐮𝐭 𝐢𝐭 𝐝𝐞𝐜𝐢𝐝𝐞𝐬 𝐲𝐨𝐮𝐫 𝐀𝐈’𝐬 𝐢𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞.pdf
 
 

Repository files navigation

Ultimate Vector Databases Landscape (2026)

A comprehensive collection of Vector Databases, Vector Search Engines, ANN Libraries, Graph Vector Databases, and Vector-Enabled Databases used in AI, LLMs, RAG, Semantic Search, Recommendation Systems, AI Agents, Memory Systems, and Generative AI applications.

Vector Databases RAG LLMs AI Agents

📚 Table of Contents

What is a Vector Database?

A Vector Database stores embeddings (vectors) generated by AI models such as:

  • OpenAI Embeddings
  • Gemini Embeddings
  • Cohere Embeddings
  • BGE Embeddings
  • E5 Embeddings
  • Sentence Transformers
  • CLIP
  • Hugging Face Embedding Models

These embeddings enable:

✅ Semantic Search

✅ Similarity Search

✅ Hybrid Search

✅ Recommendation Systems

✅ AI Agent Memory

✅ Retrieval-Augmented Generation (RAG)

✅ Knowledge Retrieval

✅ Multimodal Search

✅ Long-Term LLM Memory

Typical RAG Architecture

User Query
     │
     ▼
Embedding Model
(OpenAI / BGE / E5)
     │
     ▼
Vector Database
(Qdrant / Pinecone / Milvus)
     │
     ▼
Retriever
     │
     ▼
LLM
(GPT / Claude / Gemini / Llama)
     │
     ▼
Final Response

Open Source Vector Databases

Database Description Official Website
Milvus Distributed vector database built for billion-scale similarity search. https://milvus.io
Qdrant Rust-based production-ready vector database with advanced filtering. https://qdrant.tech
Weaviate AI-native vector database supporting hybrid search and generative AI. https://weaviate.io
ChromaDB Lightweight vector database designed for LLM applications. https://www.trychroma.com
LanceDB Embedded vector database built on Apache Arrow. https://lancedb.com
Vespa Large-scale search engine supporting vector and structured search. https://vespa.ai
Vald Kubernetes-native distributed vector search engine. https://vald.vdaas.org
Vearch Distributed vector retrieval system for AI applications. https://vearch.readthedocs.io
Epsilla Local-first AI-native vector database. https://epsilla.com
DeepLake Vector database and data lake for multimodal AI. https://activeloop.ai
Marqo Tensor search engine combining AI models and vector search. https://marqo.ai
NucliaDB Semantic search and knowledge retrieval platform. https://nuclia.com
Jina AI Search Neural search framework for multimodal AI retrieval. https://jina.ai
Pathway Vector Store Real-time vector retrieval for streaming data. https://pathway.com
Txtai Semantic search platform with vector indexing. https://github.com/neuml/txtai
USearch High-performance SIMD-optimized vector search library. https://unum-cloud.github.io/usearch
SPTAG Microsoft's ANN library for large-scale vector search. https://github.com/microsoft/SPTAG

Managed / Cloud Vector Databases

Database Description Official Website
Pinecone Fully managed serverless vector database. https://www.pinecone.io
Zilliz Cloud Managed Milvus service. https://zilliz.com/cloud
Qdrant Cloud Hosted Qdrant deployment with scaling support. https://qdrant.tech/cloud
Weaviate Cloud Managed Weaviate platform. https://weaviate.io
Astra DB Vector Search Cassandra-based vector database service. https://www.datastax.com/products/datastax-astra-db
Vertex AI Vector Search Google Cloud enterprise vector retrieval service. https://cloud.google.com/vertex-ai
Azure AI Search Microsoft enterprise search with vector support. https://azure.microsoft.com
Amazon OpenSearch Service AWS-managed OpenSearch deployment. https://aws.amazon.com/opensearch-service
MongoDB Atlas Vector Search Integrated vector search in MongoDB Atlas. https://www.mongodb.com/atlas
Neo4j AuraDB Managed graph database with vector indexing. https://neo4j.com/cloud/aura
Elastic Cloud Managed Elasticsearch service. https://www.elastic.co/cloud
Redis Cloud Managed Redis vector search service. https://redis.io/cloud
AlloyDB AI PostgreSQL-compatible AI database from Google. https://cloud.google.com/alloydb
Rockset Real-time indexing and vector search platform. https://rockset.com
SingleStore Helios Managed distributed SQL and vector database. https://www.singlestore.com

Databases with Native Vector Search

Database Description Official Website
PostgreSQL + pgvector PostgreSQL extension adding vector similarity search. https://github.com/pgvector/pgvector
MongoDB Atlas Native vector search inside document collections. https://www.mongodb.com
Redis In-memory database supporting vector indexing. https://redis.io
MySQL HeatWave MySQL with integrated vector search capabilities. https://www.mysql.com/products/heatwave
SingleStore Distributed SQL database with vector support. https://www.singlestore.com
Couchbase Multi-model database supporting vector retrieval. https://www.couchbase.com
Cassandra Distributed NoSQL database with vector extensions. https://cassandra.apache.org
ScyllaDB Cassandra-compatible high-performance vector database. https://www.scylladb.com
CockroachDB Distributed SQL database with vector indexing. https://www.cockroachlabs.com
Oracle AI Vector Search Enterprise vector search built into Oracle Database. https://www.oracle.com/database/ai-vector-search
SAP HANA Cloud Enterprise vector engine integrated with SAP ecosystem. https://www.sap.com/products/technology-platform/hana.html
TiDB Distributed SQL database supporting vector search. https://www.pingcap.com
YugabyteDB PostgreSQL-compatible distributed database. https://www.yugabyte.com
ClickHouse Analytics database supporting vector retrieval. https://clickhouse.com
DuckDB Lightweight analytics database with vector extensions. https://duckdb.org

Graph Databases with Vector Search

Database Description Official Website
Neo4j Industry-leading graph database for GraphRAG. https://neo4j.com
TigerGraph Enterprise graph analytics platform. https://www.tigergraph.com
Memgraph Real-time graph database optimized for analytics. https://memgraph.com
ArangoDB Multi-model database combining graph and vector search. https://arangodb.com
Dgraph Distributed graph database for scalable applications. https://dgraph.io
TerminusDB Knowledge graph database with semantic reasoning. https://terminusdb.com
NebulaGraph Large-scale distributed graph database. https://nebula-graph.io
HugeGraph Apache graph database for enterprise deployments. https://hugegraph.apache.org
JanusGraph Scalable graph database over distributed storage systems. https://janusgraph.org

Search Engines with Vector Search

Search Engine Description Official Website
Elasticsearch Enterprise search platform with vector search. https://www.elastic.co
OpenSearch Open-source search and analytics suite. https://opensearch.org
Apache Solr Search platform supporting vector indexing. https://solr.apache.org
Vespa Search and recommendation engine. https://vespa.ai
Lucene Vector Search Core search library powering modern search systems. https://lucene.apache.org
Typesense Developer-friendly search engine with vector support. https://typesense.org
Meilisearch Lightweight search engine with semantic search support. https://www.meilisearch.com

ANN Libraries

Library Description Official Website
FAISS Meta's industry-standard similarity search library. https://github.com/facebookresearch/faiss
HNSWlib Fast ANN library based on HNSW indexing. https://github.com/nmslib/hnswlib
Annoy Spotify's memory-efficient ANN library. https://github.com/spotify/annoy
ScaNN Google's ANN library optimized for large-scale retrieval. https://github.com/google-research/google-research/tree/master/scann
NMSLIB Flexible similarity search framework. https://github.com/nmslib/nmslib
DiskANN Microsoft's disk-based ANN search library. https://github.com/microsoft/DiskANN
FLANN ANN library popular in computer vision applications. https://www.cs.ubc.ca/research/flann
NearPy Python library for locality-sensitive hashing search. https://github.com/pixelogik/NearPy
USearch Ultra-fast vector search implementation. https://unum-cloud.github.io/usearch
SPTAG Microsoft ANN library for large-scale retrieval. https://github.com/microsoft/SPTAG

AI Agent Memory Systems

System Description Website
Mem0 Persistent memory layer for AI agents. https://mem0.ai
Letta Stateful AI agents with long-term memory. https://www.letta.com
Zep Memory platform for conversational AI. https://www.getzep.com
LangGraph Memory Agent memory framework within LangGraph. https://www.langchain.com
LlamaIndex Memory Long-term memory management for LLM apps. https://www.llamaindex.ai
Redis Memory Store Real-time memory backend for agents. https://redis.io
Qdrant Memory Systems Vector-based memory storage. https://qdrant.tech

Feature Comparison

Feature Pinecone Qdrant Milvus Weaviate ChromaDB
Open Source ❌ ✅ ✅ ✅ ✅
Cloud Hosted ✅ ✅ ✅ ✅ ❌
Hybrid Search ✅ ✅ ✅ ✅ ❌
Metadata Filtering ✅ ✅ ✅ ✅ ✅
Horizontal Scaling ✅ ✅ ✅ ✅ ❌
Multi-Tenant Support ✅ ✅ ✅ ✅ ❌
Production Ready ✅ ✅ ✅ ✅ ⚠️
Beginner Friendly ✅ ✅ ⚠️ ⚠️ ✅

Selection Guide

Use Case Recommended Solution
Learning RAG ChromaDB, FAISS
Startup MVP Qdrant, Pinecone
Production RAG Qdrant, Milvus, Pinecone
Enterprise Search Elasticsearch, OpenSearch
PostgreSQL Applications pgvector
MongoDB Applications MongoDB Atlas
Agent Memory Systems Qdrant, Weaviate, Mem0
Knowledge Graph + RAG Neo4j
Streaming Data Retrieval Pathway
Billion Scale Search Milvus, Pinecone, Vespa

Industry (2026)

Most Adopted

  1. Pinecone
  2. Qdrant
  3. Milvus
  4. Weaviate
  5. PostgreSQL + pgvector

Fastest Growing

  1. Qdrant
  2. LanceDB
  3. Marqo
  4. DeepLake
  5. Epsilla

Enterprise Favorites

  1. Azure AI Search
  2. Elasticsearch
  3. OpenSearch
  4. MongoDB Atlas
  5. Neo4j

Ecosystem Statistics

Category Count
Open Source Vector Databases 17+
Managed Vector Databases 15+
Native Vector Databases 15+
Graph Databases 9+
Search Engines 7+
ANN Libraries 10+
Memory Systems 7+

Total Ecosystem Size: 75+ Technologies

Learning Roadmap

Beginner

  • Embeddings
  • Semantic Search
  • FAISS
  • ChromaDB

Intermediate

  • RAG
  • Qdrant
  • Pinecone
  • Hybrid Search

Advanced

  • Milvus
  • Distributed Retrieval
  • Agent Memory Systems
  • GraphRAG
  • Knowledge Graphs
  • Multi-Vector Retrieval

References

  • Milvus Documentation
  • Qdrant Documentation
  • Pinecone Documentation
  • Weaviate Documentation
  • LangChain Documentation
  • LlamaIndex Documentation
  • OpenAI Embeddings Guide

Feel free to add:

  • New Vector Databases
  • ANN Libraries
  • Benchmarks
  • RAG Tutorials
  • GraphRAG Tools
  • AI Agent Memory Platforms
  • Production Case Studies

Vector Databases: from Embeddings to Applications

image image image image

𝐏𝐢𝐧𝐞𝐜𝐨𝐧𝐞: 𝐀 𝐏𝐫𝐚𝐜𝐭𝐢𝐜𝐚𝐥 𝐕𝐞𝐜𝐭𝐨𝐫 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞 𝐟𝐨𝐫 𝐒𝐞𝐦𝐚𝐧𝐭𝐢𝐜 𝐒𝐞𝐚𝐫𝐜𝐡

image

𝐏𝐢𝐧𝐞𝐜𝐨𝐧𝐞

image

VECTOR DATABASE

image
  • A Vector Database is a specialized database designed to store, index, and search vector embeddings generated from unstructured data such as text, images, audio, and videos. It enables semantic search by measuring similarity between vectors using techniques like Cosine Similarity, Euclidean Distance, and Dot Product.

📖 What is a Vector Database?

  • Traditional databases search using exact matches or structured queries. Vector databases store high-dimensional numerical representations (embeddings) and retrieve the most relevant results based on semantic similarity.

Example

Query:

How do I reset my account password?

Matched Document:

Steps to change your login credentials
  • Even though the keywords differ, the meaning is similar, allowing the vector database to retrieve the correct result.

🏗️ Core Architecture

Unstructured Data
       │
       ▼
Embedding Model
       │
       ▼
Vector Representation
       │
       ▼
Vector Index Engine
       │
       ▼
Vector Store + Metadata
       │
       ▼
Similarity Search & Filtering
       │
       ▼
Top-K Results

Components

Component Description
Unstructured Data Documents, PDFs, Images, Audio, Videos
Embedding Model Converts data into vectors
Vector Representation Numerical embedding of content
Vector Index Engine ANN-based indexing for fast retrieval
Metadata Layer Stores source information and filters
Similarity Search Finds nearest vectors
Top-K Results Returns the most relevant matches

🔍 How Vector Search Works

User Query
    │
    ▼
Embedding Model
    │
    ▼
Query Vector
    │
    ▼
Vector Database
    │
    ▼
Similarity Search
    │
    ▼
Top-K Relevant Results

📂 Types of Vector Databases

🖥️ On-Premise Vector Databases

Database Description
Chroma Python-native vector database optimized for LLM and RAG applications
Qdrant High-performance Rust-based vector engine with REST and gRPC APIs
Weaviate Hybrid search engine with GraphQL support
Milvus Distributed vector database capable of managing billions of vectors

☁️ Cloud-Hosted Vector Databases

Database Description
Redis Vector Lightweight vector search integrated into Redis ecosystem
Pinecone Fully managed cloud-native vector database
Zilliz Cloud Managed Milvus service for enterprise applications

⚡ Embedded / Lightweight Vector Stores

Database Description
Annoy Spotify's tree-based ANN search library
FAISS Meta's similarity search library with GPU acceleration
ScaNN Google's scalable nearest-neighbor search library

💡 Popular Use Cases

1️⃣ Chat Memory

Store conversation embeddings to maintain context across interactions.

Example

  • AI Assistants
  • Customer Support Bots
  • Personal Chatbots

2️⃣ Retrieval-Augmented Generation (RAG)

Retrieve relevant documents before generating responses.

User Query
     │
     ▼
Vector Search
     │
     ▼
Relevant Documents
     │
     ▼
LLM
     │
     ▼
Final Answer

Applications

  • Enterprise Knowledge Base
  • Document Q&A Systems
  • Research Assistants

3️⃣ Recommendation Systems

Recommend similar:

  • Products
  • Movies
  • Music
  • Jobs
  • Articles

based on embedding similarity.

4️⃣ Semantic Search

Search based on meaning instead of keywords.

Traditional Search

"reset password"

Semantic Search

"can't access my account"

Both return similar results.

5️⃣ Multimodal Search

Search across multiple data formats:

  • Text → Text
  • Text → Image
  • Image → Image
  • Audio → Audio

Examples:

  • Google Lens
  • Reverse Image Search
  • Multimedia Retrieval

✨ Key Features

Real-Time Updates

  • Add vectors instantly
  • Delete vectors instantly
  • Update vectors dynamically

Scalability

Supports:

  • Millions of vectors
  • Billions of vectors
  • Distributed deployments

Fast Approximate Search

Uses ANN algorithms:

  • HNSW
  • IVF
  • PQ
  • DiskANN

for millisecond-level retrieval.

High-Dimensional Storage

Supports embeddings such as:

384 Dimensions
768 Dimensions
1024 Dimensions
1536 Dimensions
3072 Dimensions

🧠 Popular Embedding Models

Model Provider
text-embedding-3-small OpenAI
text-embedding-3-large OpenAI
BGE Models BAAI
E5 Models Microsoft
Sentence Transformers Hugging Face

🔥 Vector Database in RAG Architecture

Documents
    │
    ▼
Chunking
    │
    ▼
Embedding Model
    │
    ▼
Vector Database
    │
    ▼
Similarity Search
    │
    ▼
Top-K Chunks
    │
    ▼
LLM
    │
    ▼
Generated Response

📊 Comparison

Feature Traditional DB Vector DB
Keyword Search ✅ ✅
Semantic Search ❌ ✅
Similarity Matching ❌ ✅
Embedding Storage ❌ ✅
RAG Support ❌ ✅
Multimodal Search ❌ ✅

🛠️ Best Vector Database Selection

Scenario Recommended Database
Local RAG Project Chroma
Enterprise RAG Pinecone
Open-Source Production Qdrant
Massive Scale Milvus
Research & Experiments FAISS
Redis Ecosystem Redis Vector
Managed Milvus Zilliz Cloud

Conclusion

Vector Databases are the backbone of modern AI applications, enabling:

  • Semantic Search
  • Retrieval-Augmented Generation (RAG)
  • Recommendation Systems
  • Chat Memory
  • Multimodal Search

They store embeddings efficiently and retrieve the most relevant information using similarity search, making them essential for building intelligent AI systems powered by Large Language Models (LLMs).

Vector Database cheatsheet

image image
image

📞 CONTACT & NETWORKING 📞

💼 Professional Networks

LinkedIn GitHub X Portfolio Email Medium Stack Overflow

🚀 AI/ML & Data Science AI/ML 1620+ Problem Solved

Streamlit HuggingFace Kaggle

💻 Competitive Programming Including all coding plateform's 5000+ Problems/Questions solved

LeetCode HackerRank CodeChef Codeforces GeeksforGeeks HackerEarth InterviewBit

📊 GitHub Stats & Metrics 📊

Profile Views

GitHub Streak Stats

Typing SVG

Footer Typing SVG

About

Vector Database

arize.com/ | https://www.nugget.com/

Resources

Readme

License

Apache-2.0 license
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors