Skip to content

Core Concepts Memory and Semantic Search System

Jump to bottom
github-actions[bot] edited this page Aug 3, 2026 · 3 revisions

Memory and Semantic Search System

Referenced Files in This Document

Table of Contents

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendices

Introduction

This document explains the memory and semantic search system, focusing on:

  • The memory store architecture and its adapter pattern for diverse data sources
  • Vector embedding generation using Qdrant and embedding service configuration
  • Semantic search capabilities, including hybrid search combining BM25 keyword matching with vector similarity
  • Data flow from raw content to searchable vectors, covering preprocessing, chunking, and metadata extraction
  • Examples of memory operations, search queries, and embedding configuration
  • Performance optimization, caching strategies, and scalability considerations for large datasets

The system is designed to ingest heterogeneous content via adapters, transform it into structured artifacts, generate embeddings, index them in Qdrant, and support both semantic and hybrid search across spaces and tenants.

Project Structure

At a high level, the memory and search subsystem spans several modules:

  • Memory layer: adapters, artifact handling, validation, and accessors
  • Embedding layer: provider abstraction, tokenization, and configuration
  • Qdrant integration: connection, initialization, indexing, retrieval, and search
  • Tools and HTTP endpoints: user-facing APIs for training, search, and export
  • Utilities: query building, vector management, space filtering, and tenant context
graph TB
subgraph "Memory Layer"
A["store.ts"]
B["store-methods.ts"]
C["store-init.ts"]
D["store-artifact.ts"]
E["store-adapter.ts"]
F["adapter-builder.ts"]
G["qdrant-point-to-memory.ts"]
H["memory-accessors.ts"]
I["artifact-metadata.ts"]
end
subgraph "Embedding Layer"
J["embedding/service.ts"]
K["embedding/config.ts"]
L["embedding/providers.ts"]
M["embedding/bm25-tokenizer.ts"]
N["embedding/types.ts"]
end
subgraph "Qdrant Integration"
O["qdrant/service.ts"]
P["qdrant/connection.ts"]
Q["qdrant/initialization.ts"]
R["qdrant/memory-store.ts"]
S["qdrant/memory-updates.ts"]
T["qdrant/memory-retrieval.ts"]
U["qdrant/search.ts"]
V["qdrant/types.ts"]
W["qdrant/protocol.ts"]
X["qdrant/resources.ts"]
Y["qdrant/snapshots.ts"]
Z["qdrant/utils.ts"]
end
subgraph "Tools & HTTP"
AA["tools/search.ts"]
AB["http-api-routes.ts"]
AC["http-api-train-json.ts"]
AD["cli/commands/search.ts"]
end
subgraph "Utilities"
AE["utils/qdrant-query-utils.ts"]
AF["utils/qdrant-vector-management.ts"]
AG["utils/qdrant-vector-types.ts"]
AH["utils/qdrant-collection-utils.ts"]
AI["utils/memory-body.ts"]
AJ["utils/memory-store-utils.ts"]
AK["utils/resolve-space-param.ts"]
AL["utils/space-filter.ts"]
end
A --> B
A --> C
A --> D
A --> E
A --> F
A --> G
A --> H
A --> I
A --> J
J --> K
J --> L
J --> M
J --> N
A --> O
O --> P
O --> Q
O --> R
O --> S
O --> T
O --> U
O --> V
O --> W
O --> X
O --> Y
O --> Z
AA --> A
AB --> AA
AC --> A
AD --> AA
AE --> O
AF --> O
AG --> O
AH --> O
AI --> A
AJ --> A
AK --> A
AL --> A
Loading

Diagram sources

Section sources

Core Components

  • Memory Store: Central orchestrator for memory operations, coordinating adapters, artifacts, and persistence. It provides methods for adding, updating, deleting, and searching memories.
  • Adapters: Pluggable components that read content from various sources (e.g., files, MCP tools, shell commands), normalize content, and produce artifacts suitable for indexing.
  • Artifact Pipeline: Converts normalized content into structured artifacts with metadata, text chunks, and optional title or body segments.
  • Embedding Service: Abstracts embedding providers, configures models, and exposes functions to generate vectors and BM25 tokens.
  • Qdrant Integration: Manages connections, collections, points, and performs upserts, retrieval, and search operations.
  • Hybrid Search: Combines BM25 keyword scoring with vector similarity to improve recall and precision.
  • Tools and HTTP Endpoints: Expose training, search, and export functionality to CLI and HTTP clients.

Key responsibilities:

  • Normalize and validate input content
  • Chunk and tokenize text for BM25
  • Generate embeddings via configured providers
  • Upsert points into Qdrant with rich metadata
  • Execute hybrid queries with filters by space and tenant

Section sources

Architecture Overview

The system follows a layered architecture:

  • Presentation: CLI and HTTP endpoints
  • Orchestration: Memory store and embedding service
  • Persistence: Qdrant vector database
  • Utilities: Query builders, vector utilities, space filters, and tenant context
sequenceDiagram
participant Client as "Client (CLI/HTTP)"
participant API as "HTTP/API Layer"
participant Tool as "Search Tool"
participant MemStore as "Memory Store"
participant EmbedSvc as "Embedding Service"
participant Qdrant as "Qdrant Service"
Client->>API : "POST /train or /search"
API->>Tool : "Invoke tool with schema"
Tool->>MemStore : "Build query / Index artifacts"
MemStore->>EmbedSvc : "Generate embeddings + BM25 tokens"
EmbedSvc-->>MemStore : "Vectors and tokens"
MemStore->>Qdrant : "Upsert points or execute hybrid search"
Qdrant-->>MemStore : "Results or ack"
MemStore-->>Tool : "Normalized results"
Tool-->>API : "Structured response"
API-->>Client : "JSON payload"
Loading

Diagram sources

Detailed Component Analysis

Memory Store and Adapter Pattern

The memory store coordinates ingestion and retrieval. Adapters implement a contract to read from different sources and produce standardized artifacts. An adapter builder constructs instances based on configuration.

classDiagram
class MemoryStore {
+addArtifacts()
+updateArtifacts()
+deleteArtifacts()
+searchHybrid()
+searchTitleSimilarity()
}
class StoreAdapter {
<<interface>>
+read()
+normalize()
+toArtifact()
}
class AdapterBuilder {
+build(uri)
}
class ArtifactMetadata {
+extract()
}
class QdrantPointToMemory {
+mapPointToPoint()
}
MemoryStore --> StoreAdapter : "uses"
MemoryStore --> AdapterBuilder : "constructs"
MemoryStore --> ArtifactMetadata : "reads"
MemoryStore --> QdrantPointToMemory : "maps"
Loading

Diagram sources

Key behaviors:

  • Validation: Protocol structure and markdown size limits are enforced before indexing.
  • Accessors: Provide safe reading/writing patterns and helper utilities for memory bodies.
  • Title similarity search: Dedicated path for title-based matching.

Section sources

Embedding Service and Providers

The embedding service abstracts provider-specific implementations and exposes unified methods for generating vectors and BM25 tokens. Configuration controls model selection, dimensions, and rate limiting.

flowchart TD
Start(["Input Text"]) --> Tokenize["Tokenize for BM25"]
Tokenize --> Tokens["BM25 Tokens"]
Start --> Embed["Call Provider Embedding"]
Embed --> Vector["Vector Embedding"]
Tokens --> Output["Return {tokens, vector}"]
Vector --> Output
Loading

Diagram sources

Configuration highlights:

  • Model selection and dimension alignment with Qdrant collection schema
  • Rate limiting and health checks for provider availability
  • Audit logging for embedding usage and errors

Section sources

Qdrant Integration and Indexing

Qdrant service manages connection lifecycle, collection initialization, and point operations. Memory updates and retrieval are encapsulated to ensure consistency and performance.

sequenceDiagram
participant MemStore as "Memory Store"
participant QService as "Qdrant Service"
participant Conn as "Connection"
participant Init as "Initialization"
participant Updates as "Memory Updates"
participant Retrieval as "Memory Retrieval"
participant Search as "Search"
MemStore->>QService : "Initialize collections"
QService->>Conn : "Connect"
QService->>Init : "Ensure collection schema"
MemStore->>Updates : "Upsert points"
Updates->>QService : "Batch upsert"
MemStore->>Retrieval : "Get points by IDs"
Retrieval->>QService : "Query points"
MemStore->>Search : "Execute hybrid search"
Search->>QService : "Vector + filter query"
Loading

Diagram sources

Indexing strategy:

  • Points include vector embeddings and rich metadata (space, tenant, artifact identifiers, titles, and body snippets).
  • Collections are initialized with appropriate vector sizes and distance metrics aligned with embedding providers.
  • Batch upserts optimize throughput for large datasets.

Section sources

Hybrid Search: BM25 + Vector Similarity

Hybrid search combines BM25 keyword relevance with vector similarity to balance precision and recall. Queries can be filtered by space and tenant context.

flowchart TD
QStart(["User Query"]) --> BuildQuery["Build Hybrid Query"]
BuildQuery --> BM25["BM25 Keyword Scoring"]
BuildQuery --> VectorSim["Vector Similarity Scoring"]
BM25 --> Combine["Combine Scores"]
VectorSim --> Combine
Combine --> Filter["Apply Space/Tenant Filters"]
Filter --> Rank["Rank Results"]
Rank --> QEnd(["Top-K Results"])
Loading

Diagram sources

Examples:

  • Pure vector search: Provide a query vector and top-k limit.
  • BM25-only search: Provide tokens and filters without vector.
  • Hybrid search: Provide both tokens and vector; combine scores according to weights.

Section sources

Data Flow: Raw Content to Searchable Vectors

End-to-end pipeline from ingestion to search:

sequenceDiagram
participant Source as "Data Source (Adapters)"
participant Normalizer as "Normalize & Validate"
participant Chunker as "Chunk & Tokenize"
participant Embedder as "Embedding Service"
participant Upserter as "Qdrant Upsert"
participant Retriever as "Qdrant Retrieve/Search"
Source->>Normalizer : "Raw content"
Normalizer->>Chunker : "Normalized text"
Chunker->>Embedder : "Chunks + tokens"
Embedder-->>Chunker : "Vectors + tokens"
Chunker->>Upserter : "Points with metadata"
Upserter-->>Retriever : "Indexed artifacts"
Retriever-->>Source : "Search results"
Loading

Preprocessing steps:

  • Content normalization and sanitization
  • Markdown size validation and protocol structure checks
  • Chunking strategies for optimal retrieval granularity
  • Metadata extraction (titles, identifiers, space, tenant)

Diagram sources

Section sources

Examples: Memory Operations, Search Queries, and Embedding Configuration

  • Memory operations: Add/update/delete artifacts through the memory store methods.
  • Search queries: Use the search tool to perform pure vector, BM25-only, or hybrid searches with filters.
  • Embedding configuration: Set provider parameters, model names, and dimensions in the embedding service configuration.

Reference paths:

Section sources

Dependency Analysis

High-level dependencies between core modules:

graph LR
MemStore["Memory Store"] --> EmbedSvc["Embedding Service"]
MemStore --> QdrantSvc["Qdrant Service"]
EmbedSvc --> Providers["Providers"]
QdrantSvc --> Conn["Connection"]
QdrantSvc --> Init["Initialization"]
QdrantSvc --> Updates["Memory Updates"]
QdrantSvc --> Retrieval["Memory Retrieval"]
QdrantSvc --> Search["Search"]
Tools["Search Tool"] --> MemStore
HTTP["HTTP Routes"] --> Tools
Utils["Query & Vector Utils"] --> QdrantSvc
Loading

Diagram sources

Coupling and cohesion:

  • Memory store depends on embedding and qdrant services but remains cohesive around orchestration.
  • Qdrant service encapsulates all persistence concerns, improving modularity.
  • Embedding service isolates provider logic, enabling easy replacement.

Potential circular dependencies:

  • Avoid direct imports between Qdrant and Embedding services; rely on memory store as mediator.

External integrations:

  • Qdrant client library for vector operations
  • External embedding providers (e.g., OpenAI, local models)

Interface contracts:

  • Store adapter contract defines read/normalize/toArtifact methods.
  • Embedding service contract defines embed and tokenize methods.
  • Qdrant service contract defines upsert, retrieve, and search methods.

Section sources

Performance Considerations

  • Batch upserts: Group points to reduce network overhead and increase throughput.
  • Connection pooling: Reuse Qdrant connections to minimize latency.
  • Chunk sizing: Tune chunk length to balance retrieval accuracy and storage cost.
  • BM25 tokenization: Optimize tokenizer settings for domain-specific vocabulary.
  • Filtering: Apply space and tenant filters early to reduce result set size.
  • Caching: Cache frequent search results and embedding outputs where appropriate.
  • Concurrency: Limit concurrent embedding requests to respect provider rate limits.
  • Monitoring: Track embedding latency, Qdrant query times, and error rates.

[No sources needed since this section provides general guidance]

Troubleshooting Guide

Common issues and diagnostics:

  • Embedding failures: Check provider health and configuration; review audit logs for errors.
  • Qdrant connectivity: Verify connection parameters and collection initialization status.
  • Invalid inputs: Ensure protocol structure and markdown size constraints are met.
  • Search anomalies: Inspect hybrid query construction and filter application.

Diagnostic references:

Section sources

Conclusion

The memory and semantic search system integrates a flexible adapter pattern, robust embedding generation, and powerful hybrid search backed by Qdrant. By separating concerns across memory orchestration, embedding abstraction, and persistence, the system scales to large datasets while maintaining high-quality retrieval. Proper configuration, chunking strategies, and performance tuning are essential for optimal outcomes.

[No sources needed since this section summarizes without analyzing specific files]

Appendices

API and CLI References

Section sources

Utility References

Section sources

KAIROS MCP

Clone this wiki locally