Skip to content

Testing and Quality Assurance Integration Testing

Jump to bottom
github-actions[bot] edited this page Aug 3, 2026 · 3 revisions

Integration Testing

Referenced Files in This Document

Table of Contents

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendices

Introduction

This document explains the integration testing strategy for Kairos MCP. It covers the test harness architecture, scenario-based testing approach, and how to set up tests against real dependencies such as databases, authentication services, and external APIs. It also documents utilities and helpers for creating test scenarios, examples for API endpoints, workflow execution, and multi-component interactions, and provides guidance on database state management, data isolation, cleanup procedures, performance considerations, parallel execution strategies, and maintainability best practices.

Project Structure

Integration tests live under the tests/integration directory and are organized by feature area with a shared harness for bootstrapping real dependencies. The harness manages lifecycle of external services (Keycloak, Qdrant, Redis, Postgres), application server startup, and per-test setup/teardown. Contracts define expected behavior for tools and APIs. Shared utilities provide common helpers for authentication headers, timeouts, and client utilities.

graph TB
subgraph "Tests"
A["integration/"]
B["unit/"]
C["ui/"]
D["load/"]
end
subgraph "Integration Harness"
H1["harness/index.ts"]
H2["harness/app-server.ts"]
H3["harness/db.ts"]
H4["harness/keycloak.ts"]
H5["harness/qdrant.ts"]
H6["harness/redis.ts"]
end
subgraph "Scenarios"
S1["scenarios/auth-flow.test.ts"]
S2["scenarios/workflow-execution.test.ts"]
S3["scenarios/multi-component-interaction.test.ts"]
end
subgraph "Contracts"
C1["contracts/spaces-tool.contract.ts"]
end
subgraph "Utilities"
U1["utils/auth-headers.ts"]
U2["utils/test-timeouts.ts"]
U3["utils/keycloak-container.ts"]
U4["utils/mcp-client-utils.ts"]
U5["utils/prometheus-parser.ts"]
end
A --> H1
H1 --> H2
H1 --> H3
H1 --> H4
H1 --> H5
H1 --> H6
A --> S1
A --> S2
A --> S3
A --> C1
A --> U1
A --> U2
A --> U3
A --> U4
A --> U5
Loading

Diagram sources

Section sources

Core Components

The integration test harness is responsible for:

  • Bootstrapping the application server with real configuration
  • Starting and managing external dependencies (Keycloak, Qdrant, Redis, Postgres)
  • Providing per-test fixtures and teardown logic
  • Exposing typed helpers for HTTP and MCP clients
  • Enforcing contracts for tool responses and behaviors

Key responsibilities:

  • Application server lifecycle: start, stop, health checks
  • Dependency lifecycle: containerized or local service management
  • Test data seeding and isolation: ensure deterministic state per test
  • Authentication flow: obtain tokens and attach headers
  • Contract validation: assert schema and behavior consistency

Section sources

Architecture Overview

The integration test suite orchestrates multiple components:

  • Test runner (Jest) executes scenario files
  • Harness initializes the app server and external services
  • Scenarios drive workflows via HTTP and MCP clients
  • Contracts validate tool outputs and schemas
  • Utilities assist with auth, timeouts, and parsing metrics
sequenceDiagram
participant Runner as "Jest Runner"
participant Harness as "Test Harness"
participant App as "App Server"
participant KC as "Keycloak"
participant DB as "Postgres/Qdrant/Redis"
participant Client as "HTTP/MCP Client"
Runner->>Harness : "Initialize environment"
Harness->>DB : "Start containers / connect"
Harness->>KC : "Start Keycloak"
Harness->>App : "Start app server"
App-->>Harness : "Health OK"
Runner->>Client : "Run scenario"
Client->>KC : "Authenticate"
KC-->>Client : "Token"
Client->>App : "Call API / MCP tool"
App->>DB : "Read/Write state"
DB-->>App : "Result"
App-->>Client : "Response"
Client-->>Runner : "Assertions pass/fail"
Runner->>Harness : "Teardown"
Harness->>App : "Stop server"
Harness->>KC : "Stop Keycloak"
Harness->>DB : "Cleanup resources"
Loading

Diagram sources

Detailed Component Analysis

Test Harness Lifecycle

The harness centralizes dependency management and server lifecycle. It exposes methods to start services, wait for readiness, and perform cleanup. Tests import the harness to bootstrap their environment deterministically.

classDiagram
class TestHarness {
+startServices() Promise~void~
+waitForReady() Promise~void~
+stopServices() Promise~void~
+getBaseUrl() string
+getClientOptions() object
}
class AppServer {
+start() Promise~void~
+healthCheck() Promise~boolean~
+stop() Promise~void~
}
class KeycloakService {
+start() Promise~void~
+ready() Promise~boolean~
+stop() Promise~void~
+getToken(clientId, secret) Promise~string~
}
class DatabaseService {
+connect() Promise~void~
+seedData(data) Promise~void~
+reset() Promise~void~
+disconnect() Promise~void~
}
class QdrantService {
+initCollections() Promise~void~
+clearVectors() Promise~void~
}
class RedisService {
+connect() Promise~void~
+flushKeys(prefix) Promise~void~
+disconnect() Promise~void~
}
TestHarness --> AppServer : "manages"
TestHarness --> KeycloakService : "manages"
TestHarness --> DatabaseService : "manages"
TestHarness --> QdrantService : "manages"
TestHarness --> RedisService : "manages"
Loading

Diagram sources

Section sources

Scenario-Based Testing Approach

Scenarios encapsulate end-to-end flows such as authentication, workflow execution, and multi-component interactions. Each scenario imports the harness and uses typed helpers to interact with the system.

flowchart TD
Start(["Scenario Entry"]) --> Setup["Setup Dependencies<br/>and Seed Data"]
Setup --> Auth["Authenticate via Keycloak"]
Auth --> CallAPI["Invoke API Endpoints"]
CallAPI --> ValidateContract["Validate Tool Contracts"]
ValidateContract --> Cleanup["Reset State and Teardown"]
Cleanup --> End(["Scenario Exit"])
Loading

Diagram sources

Section sources

API Endpoint Testing

HTTP API tests use shared helpers to construct requests, attach authentication headers, and assert responses. They cover success paths, error handling, and contract compliance.

sequenceDiagram
participant Test as "Test Case"
participant Helpers as "HTTP Helpers"
participant App as "App Server"
participant KC as "Keycloak"
Test->>Helpers : "Build request with auth headers"
Helpers->>KC : "Obtain token"
KC-->>Helpers : "Token"
Helpers->>App : "Send HTTP request"
App-->>Helpers : "Response"
Helpers-->>Test : "Assert status/body/schema"
Loading

Diagram sources

Section sources

Workflow Execution Testing

Workflow tests exercise multi-step processes such as activation, forward, and reward operations. They validate state transitions and outputs across components.

sequenceDiagram
participant Test as "Workflow Test"
participant Client as "MCP/HTTP Client"
participant App as "App Server"
participant Store as "Memory/Storage"
Test->>Client : "Begin workflow"
Client->>App : "Activate step"
App->>Store : "Persist state"
Store-->>App : "State updated"
App-->>Client : "Next action"
Test->>Client : "Forward continue"
Client->>App : "Submit solution"
App->>Store : "Record result"
Store-->>App : "Success"
App-->>Client : "Reward/Outcome"
Test-->>Test : "Assert final state"
Loading

Diagram sources

Section sources

Multi-Component Interaction Testing

These tests verify cross-service interactions including caching, search retrieval, and audit logging. They often combine HTTP calls with direct storage checks.

flowchart TD
A["Trigger Action"] --> B["Cache Invalidation"]
B --> C["Search Index Update"]
C --> D["Audit Log Emit"]
D --> E["Verify Metrics"]
E --> F["Assert Consistency"]
Loading

Diagram sources

Section sources

MCP Client Utilities

MCP-specific helpers simplify tool invocation, schema validation, and response parsing. They integrate with the harness to reuse authenticated sessions.

classDiagram
class MCPClientUtils {
+listTools() Promise~Array~
+invokeTool(name, params) Promise~any~
+validateSchema(toolName, input) boolean
}
class AuthHeaders {
+getBearerToken() Promise~string~
+attachToRequest(req) Request
}
MCPClientUtils --> AuthHeaders : "uses"
Loading

Diagram sources

Section sources

Contract Validation

Contracts define expected shapes and behaviors for tools and APIs. Tests assert that implementations adhere to these contracts, ensuring parity across interfaces.

flowchart TD
I["Input Payload"] --> V["Validate Schema"]
V --> R["Execute Tool"]
R --> O["Output Payload"]
O --> C["Assert Contract"]
C --> P["Pass/Fail"]
Loading

Diagram sources

Section sources

Dependency Analysis

Integration tests depend on:

  • External services: Keycloak, Postgres, Qdrant, Redis
  • Application server: started by harness
  • Utilities: auth headers, timeouts, MCP client helpers
  • Contracts: tool/API shape definitions
graph TB
T["Test Cases"] --> H["Harness"]
H --> KS["Keycloak Service"]
H --> PS["Postgres Service"]
H --> QS["Qdrant Service"]
H --> RS["Redis Service"]
T --> AU["Auth Headers"]
T --> MU["MCP Client Utils"]
T --> CT["Contracts"]
Loading

Diagram sources

Section sources

Performance Considerations

  • Use connection pooling for databases and caches to reduce overhead
  • Reuse authenticated sessions within a test file when safe
  • Minimize seed data size; only include necessary records
  • Prefer targeted queries over full scans during assertions
  • Leverage in-memory stores where acceptable for speed
  • Avoid unnecessary restarts of services between tests
  • Use timeouts judiciously to fail fast without flakiness

[No sources needed since this section provides general guidance]

Troubleshooting Guide

Common issues and resolutions:

  • Service readiness: Ensure health checks pass before invoking APIs
  • Authentication failures: Verify Keycloak realm and client credentials
  • Data leakage: Reset state between tests using dedicated cleanup routines
  • Timeouts: Increase test timeouts for slow operations and adjust globally if needed
  • Metrics parsing: Validate Prometheus scrape endpoints and metric names

Section sources

Conclusion

The integration testing framework for Kairos MCP provides a robust foundation for validating system behavior with real dependencies. By leveraging the harness, contracts, and utilities, teams can write maintainable, reliable tests that cover API endpoints, workflow execution, and multi-component interactions. Adhering to the guidelines in this document will improve test stability, performance, and clarity.

[No sources needed since this section summarizes without analyzing specific files]

Appendices

Setting Up Integration Tests with Real Dependencies

  • Configure environment variables for service URLs and credentials
  • Initialize Keycloak realms and users via global setup hooks
  • Prepare database schemas and seed initial data
  • Start Qdrant collections and Redis keyspace prefixes
  • Launch the application server and verify health endpoints

Section sources

Creating Test Scenarios

  • Import the harness and initialize services
  • Authenticate and obtain bearer tokens
  • Build requests using HTTP helpers or MCP client utilities
  • Assert responses against contracts and business rules
  • Clean up state and release resources

Section sources

Database State Management and Isolation

  • Use transactional resets or dedicated test databases
  • Prefix keys in Redis to isolate cache state
  • Clear Qdrant vectors per test run or namespace
  • Ensure deterministic ordering for writes affecting search results

Section sources

Parallel Test Execution Strategies

  • Partition tests by feature area to minimize contention
  • Use unique identifiers for test data to avoid collisions
  • Limit concurrency for resource-heavy tests
  • Monitor metrics and logs for bottlenecks

Section sources

KAIROS MCP

Clone this wiki locally