-
Notifications
You must be signed in to change notification settings - Fork 0
Architecture Overview
- Introduction
- Project Structure
- Core Components
- Architecture Overview
- Detailed Component Analysis
- Dependency Analysis
- Performance Considerations
- Troubleshooting Guide
- Conclusion
- Appendices
This document describes the KAIROS MCP system architecture. It covers the high-level design with the Express HTTP server, the MCP protocol layer, REST API endpoints, React web UI, and CLI interface. It documents the three main execution phases—activate (semantic matching), forward (layer execution), and reward (performance evaluation)—and explains the persistent memory architecture using Qdrant vector storage and optional Redis cache. It also details the authentication flow with Keycloak OIDC integration and bearer token validation, system boundaries, component interactions, and data flows between services. Finally, it outlines the technology stack, scalability considerations, deployment topology options, and infrastructure requirements.
The system is organized around a Node.js/TypeScript backend with Express HTTP server, an MCP protocol layer, a React SPA frontend, and a CLI. Persistent memory is backed by Qdrant with optional Redis caching. Authentication integrates with Keycloak via OIDC.
graph TB
subgraph "Runtime"
A["Express HTTP Server<br/>src/http/http-server.ts"]
B["MCP Protocol Layer<br/>src/server.ts"]
C["Qdrant Vector Store<br/>src/services/memory/store.ts"]
D["React Web UI<br/>src/ui/main.tsx"]
E["CLI Interface<br/>src/cli/index.ts"]
end
subgraph "External Services"
F["Keycloak OIDC<br/>compose.yaml"]
G["Qdrant Vector DB<br/>compose.yaml"]
H["Valkey/Redis Cache<br/>compose.yaml"]
end
A --> B
A --> C
A --> D
A --> E
B --> C
A --> F
A --> G
A --> H
Diagram sources
- src/http/http-server.ts:22-58
- src/server.ts:125-193
- src/services/memory/store.ts:20-53
- src/ui/main.tsx:1-20
- src/cli/index.ts:1-11
- compose.yaml:10-183
Section sources
- src/index.ts:74-139
- src/http/http-server.ts:22-58
- src/server.ts:125-193
- src/services/memory/store.ts:20-53
- src/ui/main.tsx:1-20
- src/cli/index.ts:1-11
- compose.yaml:10-183
- Express HTTP server: Hosts REST endpoints, MCP handler, UI static assets, health checks, and exports. Mounted in src/http/http-server.ts:22-58.
- MCP protocol layer: Registers tools (activate, forward, train, reward, tune, delete, export, spaces) and UI resources. Created in src/server.ts:125-193.
- Qdrant memory store: Provides vector search, adapter/artifact storage, and retrieval. Implemented in src/services/memory/store.ts:20-152.
- React UI: SPA entry point and routing for protocol editing, runs, and account pages. Entry in src/ui/main.tsx:1-20.
- CLI: Command-line interface for login, serving, training, exporting, and other operations. Entry in src/cli/index.ts:1-11.
- Authentication: Session-based browser auth and Bearer token validation via Keycloak/JWKS. Middleware in src/http/http-auth-middleware.ts:167-313; token validation in src/http/bearer-validate.ts:120-208.
Section sources
- src/http/http-server.ts:22-58
- src/server.ts:42-108
- src/services/memory/store.ts:20-152
- src/ui/main.tsx:1-20
- src/cli/index.ts:1-11
- src/http/http-auth-middleware.ts:167-313
- src/http/bearer-validate.ts:120-208
High-level architecture showing the Express server, MCP protocol layer, REST endpoints, React UI, CLI, and persistent memory with optional Redis cache.
graph TB
subgraph "Client Layer"
U["Browser (React UI)<br/>src/ui/main.tsx"]
CLI["CLI Tool<br/>src/cli/index.ts"]
end
subgraph "Application Server"
HTTP["Express HTTP Server<br/>src/http/http-server.ts"]
MCP["MCP Protocol Layer<br/>src/server.ts"]
AUTH["Auth Middleware<br/>src/http/http-auth-middleware.ts"]
API["REST API Routes<br/>src/http/http-api-routes.ts"]
end
subgraph "Data & Cache"
QD["Qdrant Vector DB<br/>compose.yaml"]
KV["Valkey/Redis Cache<br/>compose.yaml"]
end
subgraph "Identity"
KC["Keycloak OIDC<br/>compose.yaml"]
end
U --> HTTP
CLI --> HTTP
HTTP --> AUTH
HTTP --> MCP
HTTP --> API
MCP --> QD
API --> QD
HTTP --> QD
HTTP --> KV
AUTH --> KC
Diagram sources
- src/http/http-server.ts:22-58
- src/server.ts:125-193
- src/http/http-auth-middleware.ts:167-313
- src/http/http-api-routes.ts:22-36
- compose.yaml:10-183
The Express server initializes middleware, registers well-known endpoints, CORS, authentication, health checks, API routes, MCP handler, UI static assets, and error handlers. It starts on the configured port and delegates to the MCP server.
sequenceDiagram
participant Client as "Client"
participant HTTP as "Express HTTP Server"
participant Auth as "Auth Middleware"
participant API as "REST API Routes"
participant MCP as "MCP Handler"
participant UI as "UI Static"
Client->>HTTP : HTTP request
HTTP->>Auth : Apply auth middleware
Auth-->>HTTP : Authenticated or 401
HTTP->>API : Route to REST endpoint
HTTP->>MCP : Route to MCP handler
HTTP->>UI : Serve SPA static assets
UI-->>Client : React SPA
API-->>Client : JSON responses
MCP-->>Client : JSON-RPC over HTTP
Diagram sources
- src/http/http-server.ts:22-58
- src/http/http-auth-middleware.ts:167-313
- src/http/http-api-routes.ts:22-36
Section sources
- src/http/http-server.ts:22-58
Authentication supports two modes:
- Browser session: Cookie-based session with RP-initiated logout support.
- Bearer tokens: OIDC access tokens validated via JWKS from Keycloak.
sequenceDiagram
participant Client as "Client"
participant HTTP as "Express HTTP Server"
participant Auth as "Auth Middleware"
participant KC as "Keycloak OIDC"
participant JWKS as "JWKS Endpoint"
Client->>HTTP : GET /ui or /api/*
HTTP->>Auth : authMiddleware()
alt Session present and valid
Auth-->>HTTP : req.auth populated
HTTP-->>Client : Proceed
else Bearer token present
Auth->>KC : Resolve issuer/internal URL
Auth->>JWKS : Fetch certs
JWKS-->>Auth : JWK Set
Auth->>Auth : Validate token (iss, aud, exp, sig)
alt Valid
Auth-->>HTTP : req.auth populated
HTTP-->>Client : Proceed
else Invalid/expired
Auth-->>Client : 401 + WWW-Authenticate
end
else No auth
Auth-->>Client : 401 or redirect (browser GET)
end
Diagram sources
- src/http/http-auth-middleware.ts:167-313
- src/http/bearer-validate.ts:120-208
Section sources
- src/http/http-auth-middleware.ts:167-313
- src/http/bearer-validate.ts:120-208
The MCP server registers tools and UI resources. Tools include activate, forward, train, reward, tune, delete, export, and spaces. The server exposes capabilities and resources for UI presentation.
classDiagram
class McpServer {
+registerTool(name, metadata, handler)
+server.setRequestHandler(...)
}
class ActivateTool {
+execute(input) ActivateOutput
}
class ForwardTool {
+execute(input) ForwardOutput
}
class RewardTool {
+execute(input) RewardOutput
}
class MemoryQdrantStore {
+init()
+searchMemories(query, limit)
+getMemory(uuid)
}
McpServer --> ActivateTool : "registers"
McpServer --> ForwardTool : "registers"
McpServer --> RewardTool : "registers"
ActivateTool --> MemoryQdrantStore : "uses"
ForwardTool --> MemoryQdrantStore : "uses"
RewardTool --> MemoryQdrantStore : "uses"
Diagram sources
- src/server.ts:42-108
- src/tools/activate.ts:236-283
- src/tools/forward.ts:317-318
- src/tools/reward.ts:112-155
- src/services/memory/store.ts:20-53
Section sources
- src/server.ts:125-193
- src/tools/activate.ts:236-283
- src/tools/forward.ts:317-318
- src/tools/reward.ts:112-155
The activate tool performs semantic search to find the best adapter for the current input and returns ranked choices with next actions and optional linked artifacts.
flowchart TD
Start(["activate input"]) --> ExecId["Compute execution_id"]
ExecId --> Search["executeSearch()<br/>Qdrant similarity + rerank"]
Search --> Map["mapSearchToActivate()<br/>normalize URIs, build choices"]
Map --> Artifacts["Resolve linked artifacts<br/>mint download capabilities"]
Artifacts --> Output["Return ActivateOutput"]
Diagram sources
- src/tools/activate.ts:208-234
- src/tools/activate.ts:43-206
Section sources
- src/tools/activate.ts:208-234
The forward tool executes the first or next adapter layer, handles proof-of-work challenges, updates execution traces, and advances to subsequent layers.
sequenceDiagram
participant Client as "Client"
participant MCP as "MCP Server"
participant Forward as "forward.execute()"
participant Store as "MemoryQdrantStore"
participant Trace as "executionTraceStore"
participant PoW as "proof-of-work-store"
Client->>MCP : forward(uri, solution?)
MCP->>Forward : executeForward(...)
Forward->>Store : load layer/memory
alt First call (no solution)
Forward-->>Client : Current layer view
else Has solution
Forward->>PoW : validate/submit proof
Forward->>Trace : append execution trace
Forward-->>Client : Next layer view or final
end
Diagram sources
- src/tools/forward.ts:93-317
- src/services/execution-trace-store.ts
- src/services/proof-of-work-store.ts
Section sources
- src/tools/forward.ts:93-317
The reward tool records outcomes and scores, computes metrics, and persists evaluation metadata.
flowchart TD
RStart["reward input (layer URI, outcome, score?)"] --> Parse["Parse and validate URI"]
Parse --> Eval["evaluateReward()<br/>compute normalized score"]
Eval --> Metrics["applyRewardMetrics()<br/>update Qdrant metrics"]
Metrics --> Trace["executionTraceStore.setReward()"]
Trace --> ROutput["Return RewardOutput"]
Diagram sources
- src/tools/reward.ts:27-110
- src/services/reward-evals.ts
- src/services/reward-metrics.ts
Section sources
- src/tools/reward.ts:27-110
- Qdrant vector store: Central memory for adapters, artifacts, and training data. Provides health checks, search, and retrieval. See src/services/memory/store.ts:20-152.
- Optional Redis cache: Used for key-value caching and pub/sub (Valkey in Compose). See compose.yaml:11-31.
- Startup bootstrapping: On startup, the application waits for Qdrant readiness, probes embedding dimension, initializes memory store, triggers optional snapshot, and injects embedded MCP resources. See src/index.ts:74-139.
graph LR
App["KAIROS App<br/>src/index.ts"] --> Qdrant["Qdrant Service<br/>src/services/qdrant/index.ts"]
App --> Store["MemoryQdrantStore<br/>src/services/memory/store.ts"]
Store --> QdrantClient["@qdrant/js-client-rest"]
App --> Redis["Valkey/Redis Cache<br/>compose.yaml"]
App --> Boot["Embedded Resources Injection<br/>src/resources/mem-resources-boot.ts"]
Diagram sources
- src/index.ts:74-139
- src/services/memory/store.ts:20-53
- compose.yaml:11-31
Section sources
- src/index.ts:74-139
- src/services/memory/store.ts:20-152
- compose.yaml:11-31
The Express server mounts REST endpoints for training, activation, forwarding, reward, updates, deletion, dumping, and spaces. See src/http/http-api-routes.ts:22-36.
graph TB
API["/api/* Routes<br/>src/http/http-api-routes.ts"] --> Train["POST /api/train-*"]
API --> Activate["POST /api/activate"]
API --> Forward["POST /api/forward"]
API --> Reward["POST /api/reward"]
API --> Update["PATCH /api/update"]
API --> Delete["DELETE /api/delete"]
API --> Dump["GET /api/dump"]
API --> Spaces["GET /api/spaces"]
Diagram sources
- src/http/http-api-routes.ts:22-36
Section sources
- src/http/http-api-routes.ts:22-36
The React SPA is bootstrapped in src/ui/main.tsx:1-20 with TanStack Query for data fetching and theme provider. It integrates with the backend via REST and MCP endpoints.
graph TB
UI["React SPA<br/>src/ui/main.tsx"] --> Query["TanStack Query"]
UI --> Theme["Theme Provider"]
UI --> Pages["Pages & Components"]
Pages --> API["REST Calls"]
Pages --> MCP["MCP JSON-RPC"]
Diagram sources
- src/ui/main.tsx:1-20
Section sources
- src/ui/main.tsx:1-20
The CLI entry point is defined in src/cli/index.ts:1-11. It provides commands for login, serving, training, exporting, and other operations.
Section sources
- src/cli/index.ts:1-11
Technology stack and key dependencies:
- Node.js runtime and TypeScript
- Express for HTTP server
- MCP SDK for protocol handling
- Qdrant JS client for vector operations
- React, TanStack Query, and related UI libraries
- Redis/Valkey for caching
- Keycloak for OIDC
graph TB
Node["Node.js Runtime"] --> Express["Express"]
Express --> MCP["@modelcontextprotocol/sdk"]
Express --> Qdrant["@qdrant/js-client-rest"]
Express --> Redis["redis"]
Express --> React["React + TanStack Query"]
Express --> JOSE["jose (JWT)"]
Express --> OIDC["Keycloak OIDC"]
Diagram sources
- package.json:117-148
Section sources
- package.json:117-148
- Concurrency and rate limiting: Express rate limiter is included in dependencies; consider applying per-endpoint limits for MCP and API routes.
- Embedding dimension probing and Qdrant health checks occur at startup to ensure alignment and availability.
- Qdrant search parameters (overfetch, max fetch, group collapse) are configurable and influence latency and recall.
- Redis cache can reduce repeated reads and accelerate hot paths; ensure proper TTL and eviction policies.
- Horizontal scaling: The Helm chart supports replicas and HPA/VPA; ensure stateless HTTP layer and shared Qdrant/Redis.
[No sources needed since this section provides general guidance]
Common areas to inspect:
- Authentication: 401 responses with WWW-Authenticate indicate missing or invalid Bearer tokens or misconfiguration of trusted issuers/audiences. See src/http/http-auth-middleware.ts:248-282 and src/http/bearer-validate.ts:120-208.
- Qdrant readiness: Startup waits for health checks; timeouts indicate network or credentials issues. See src/index.ts:44-67 and src/services/memory/store.ts:59-121.
- MCP tool validation: Input schema mismatches return structured validation errors; review tool schemas and inputs. See src/tools/activate.ts:257-263, src/tools/forward.ts:317-318, src/tools/reward.ts:129-135.
Section sources
- src/http/http-auth-middleware.ts:248-282
- src/http/bearer-validate.ts:120-208
- src/index.ts:44-67
- src/services/memory/store.ts:59-121
- src/tools/activate.ts:257-263
- src/tools/forward.ts:317-318
- src/tools/reward.ts:129-135
KAIROS MCP integrates an Express HTTP server with an MCP protocol layer, a React UI, and a CLI, backed by Qdrant vector storage and optional Redis caching. The system supports robust OIDC-based authentication and provides a deterministic workflow across three phases: activate, forward, and reward. The architecture is designed for scalability via horizontal replication and shared infrastructure, with clear separation of concerns and modular components.
[No sources needed since this section summarizes without analyzing specific files]
- Docker Compose: Includes Qdrant, optional Valkey/Redis, Keycloak, and the application service. See compose.yaml:10-183.
- Kubernetes/Helm: Chart supports replicas, HPA/VPA, gateway/route exposure, and optional Keycloak/Postgres/Redis clusters. See helm/kairos-mcp/values.yaml:39-279.
Section sources
- compose.yaml:10-183
- helm/kairos-mcp/values.yaml:39-279