Skip to content

Testing and Quality Assurance Continuous Integration and Automation

Jump to bottom
github-actions[bot] edited this page Aug 3, 2026 · 3 revisions

Continuous Integration and Automation

Referenced Files in This Document

Update Summary

Changes Made

  • Updated build configuration section to reflect removal of npm scripts referencing deleted sync-kairos-install-references.py script
  • Enhanced GitHub Actions workflow documentation for wiki synchronization improvements
  • Updated configuration files section with knip.config.ts and eslint/flat-config.cjs adjustments
  • Streamlined CI pipeline documentation to reflect build configuration optimizations

Table of Contents

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendices

Introduction

This document explains the continuous integration setup and automation workflows for Kairos MCP. It covers GitHub Actions configuration for automated testing, building, and deployment; parallel test execution strategy and performance optimizations; pre-commit hooks with Husky; CI pipeline stages, job dependencies, and artifact management; security scanning and compliance checks; caching strategies; debugging guidance; and examples of custom CI scripts and automation tasks.

The CI/CD pipeline has been significantly streamlined through consolidation of three separate integration workflows into a single unified integration.yml workflow, implementing a new two-phase testing strategy with read-only-first fail-fast behavior for improved efficiency and faster feedback loops. Recent updates have further optimized the build configuration by removing deprecated npm scripts and enhancing GitHub Actions workflows for wiki synchronization.

Project Structure

The CI and automation surface is composed of:

  • GitHub Actions workflows under .github/workflows with consolidated integration testing
  • Custom CI scripts under scripts
  • Test orchestration and reporting under tests
  • Containerization and local dev tooling via Dockerfiles and compose files
  • Pre-commit hooks under .husky
  • Security scanning configuration (.trivyignore)
  • Dependency updates via Dependabot
  • Build configuration files (knip.config.ts, eslint/flat-config.cjs)
graph TB
subgraph "CI Orchestration"
GH["GitHub Actions"]
W1[".github/workflows/ci.yml"]
W2[".github/workflows/integration.yml"]
W3[".github/workflows/codeql-analysis.yml"]
DEP[".github/dependabot.yml"]
end
subgraph "Scripts"
PAR["scripts/ci-parallel-checks.mjs"]
SUM["scripts/ci-github-step-summary.mjs"]
TGZ["scripts/ci-test-tgz-install.mjs"]
INFRA["scripts/ci-wait-for-infra.sh"]
HELM["scripts/test-helm.sh"]
SNAP_I["scripts/import-test-snapshot.sh"]
SNAP_S["scripts/seed-test-snapshot.sh"]
BUILD_E["scripts/build-embed-docs.ts"]
VITE_D["scripts/build-vite-ui-env-define.ts"]
RUN_ENV["scripts/deploy-run-env.sh"]
ENV_C["scripts/env/create-env.sh"]
STDIO_EP["scripts/stdio/entrypoint.sh"]
end
subgraph "Tests"
JCFG["jest.config.js"]
VCFG["vitest.config.ts"]
SEQ["tests/jest-sequencer.cjs"]
RPT["tests/reporters/jest-github-summary-reporter.cjs"]
end
subgraph "Containers"
DKR["Dockerfile"]
DKRD["Dockerfile.dev"]
DKRS["Dockerfile.stdio"]
CMP["compose.yaml"]
end
subgraph "Hooks & Security"
HUSKY[".husky/pre-commit"]
TRIVY[".trivyignore"]
end
subgraph "Build Config"
KNIP["knip.config.ts"]
ESLINT["eslint/flat-config.cjs"]
PKG["package.json"]
end
GH --> W1
GH --> W2
GH --> W3
W1 --> PAR
W1 --> SUM
W1 --> TGZ
W1 --> INFRA
W1 --> HELM
W1 --> SNAP_I
W1 --> SNAP_S
W1 --> BUILD_E
W1 --> VITE_D
W1 --> RUN_ENV
W1 --> ENV_C
W1 --> STDIO_EP
W1 --> JCFG
W1 --> VCFG
W1 --> SEQ
W1 --> RPT
W1 --> DKR
W1 --> DKRD
W1 --> DKRS
W1 --> CMP
W1 --> HUSKY
W1 --> TRIVY
W1 --> KNIP
W1 --> ESLINT
W1 --> PKG
GH --> DEP
Loading

Diagram sources

Section sources

Core Components

  • Consolidated CI workflow orchestrator: Streamlined workflow that combines previously separate integration tests into unified pipeline with optimized job orchestration, matrix builds, caching, artifacts, and step summaries.
  • Two-phase testing strategy: New fail-fast approach that executes read-only tests first, followed by write operations only if read-only phase succeeds, improving feedback speed and resource utilization.
  • Parallel test executor: Splits suites across workers to maximize throughput with enhanced distribution logic.
  • Reporting and summaries: Produces GitHub-friendly summaries and test reports with consolidated results from unified workflow.
  • Infrastructure provisioning: Waits for external services (e.g., Keycloak, Redis, Qdrant) before running tests.
  • Helm chart validation: Runs chart linting and tests.
  • Snapshot import/seed: Prepares deterministic test data.
  • Build helpers: UI environment definition and embedded docs build.
  • Streamlined build configuration: Optimized npm scripts with removal of deprecated references and enhanced wiki synchronization.
  • Container images: Multi-stage builds for app, dev, and stdio variants.
  • Local automation: Husky pre-commit hook to enforce quality gates locally.
  • Security scanning: Trivy vulnerability scanning with ignore rules.
  • Dependency updates: Dependabot configuration for automated PRs.
  • Enhanced configuration management: Updated knip.config.ts and eslint/flat-config.cjs for improved code analysis and linting.

Section sources

Architecture Overview

The CI architecture coordinates multiple jobs that share caches and artifacts with a streamlined workflow structure. Jobs are grouped into logical stages: prepare, build, test (two-phase), package, and deploy. Matrix strategies run subsets in parallel with optimized resource allocation. Artifacts are uploaded for later jobs or manual inspection.

graph TB
A["Workflow Trigger<br/>push/pull_request/release"] --> B["Job: Prepare<br/>Install deps, cache restore"]
B --> C["Job: Build<br/>TypeScript compile, UI build"]
C --> D["Job: Lint & Security Scan<br/>Trivy, CodeQL"]
C --> E["Job: Phase 1 - Read-Only Tests<br/>Fast fail-fast execution"]
E --> F{"Phase 1 Results"}
F --> |Success| G["Job: Phase 2 - Write Operations<br/>Full integration tests"]
F --> |Failure| H["Skip Phase 2<br/>Fail fast"]
C --> I["Job: Helm Tests<br/>Lint and validate charts"]
E --> J["Upload: Test Reports"]
G --> J
I --> J
D --> K["Publish: Security Findings"]
J --> L["Job: Summary<br/>Aggregate results"]
K --> L
Loading

Diagram sources

Detailed Component Analysis

Consolidated GitHub Actions Workflow Configuration

Updated The CI workflow has been significantly streamlined through consolidation of three separate integration workflows into a single unified integration.yml workflow, with recent enhancements for wiki synchronization and build configuration optimization.

  • Unified Workflow: Single integration.yml replaces separate Integration, Integration Simple, and Integration Stdio workflows
  • Two-Phase Testing Strategy:
    • Phase 1: Read-only tests execute first with fail-fast behavior
    • Phase 2: Write operation tests only run if Phase 1 succeeds
  • Optimized Job Dependencies: Streamlined dependency chain reduces workflow complexity
  • Enhanced Matrix Strategy: Consolidated matrix configurations for better resource utilization
  • Improved Artifact Management: Centralized artifact handling across all test phases
  • Wiki Synchronization Enhancement: Improved GitHub Actions workflow for automated wiki updates
sequenceDiagram
participant GH as "GitHub Actions"
participant PREP as "Prepare Job"
participant BUILD as "Build Job"
participant PHASE1 as "Phase 1 : Read-Only Tests"
participant PHASE2 as "Phase 2 : Write Operations"
participant SEC as "Security Scan Job"
participant SUM as "Summary Job"
participant WIKI as "Wiki Sync"
GH->>PREP : "Start"
PREP-->>GH : "Cache restored"
GH->>BUILD : "Depends on Prepare"
BUILD-->>GH : "Artifacts uploaded"
GH->>PHASE1 : "Depends on Build"
GH->>SEC : "Depends on Build"
GH->>WIKI : "Sync Wiki Content"
PHASE1-->>GH : "Read-only test results"
alt Phase 1 Success
GH->>PHASE2 : "Execute write operations"
PHASE2-->>SUM : "Write test results"
else Phase 1 Failure
GH->>SUM : "Skip Phase 2, fail fast"
end
SEC-->>SUM : "Security findings"
WIKI-->>SUM : "Wiki sync status"
SUM-->>GH : "Step Summary"
Loading

Diagram sources

Section sources

Enhanced Parallel Test Execution Strategy

Updated The parallel test execution strategy now operates within the unified workflow with improved distribution logic and two-phase execution model.

  • Unified Distribution: Single script manages test suite distribution across both phases
  • Phase-Aware Partitioning: Read-only tests partitioned separately from write operations
  • Enhanced Concurrency Control: Improved worker allocation based on test type and resource requirements
  • Optimized Cache Sharing: Better cache utilization between phases and workers
  • Fail-Fast Integration: Immediate failure propagation from Phase 1 prevents unnecessary Phase 2 execution
flowchart TD
Start(["Start Unified Workflow"]) --> Discover["Discover Test Suites"]
Discover --> Phase1Partition["Partition Read-Only Tests"]
Phase1Partition --> RunPhase1["Execute Phase 1 Workers"]
RunPhase1 --> CheckResults{"Phase 1 Results"}
CheckResults --> |All Pass| Phase2Partition["Partition Write Operation Tests"]
CheckResults --> |Any Fail| SkipPhase2["Skip Phase 2 - Fail Fast"]
Phase2Partition --> RunPhase2["Execute Phase 2 Workers"]
RunPhase2 --> Collect["Collect All Reports"]
SkipPhase2 --> Collect
Collect --> Summarize["Generate Step Summary"]
Summarize --> End(["Exit with Status"])
Loading

Diagram sources

Section sources

Streamlined Build Configuration

Updated Build configuration has been optimized with removal of deprecated npm scripts and enhanced wiki synchronization capabilities.

  • Removed Deprecated Scripts: Eliminated npm scripts referencing deleted sync-kairos-install-references.py script
  • Enhanced Wiki Synchronization: Improved GitHub Actions workflow for automated wiki content updates
  • Configuration Optimization: Updated knip.config.ts and eslint/flat-config.cjs for better code analysis
  • Simplified Build Pipeline: Reduced complexity while maintaining functionality
flowchart TD
BuildStart(["Build Process"]) --> CleanConfig["Clean Build Configuration"]
CleanConfig --> RemoveDeprecated["Remove Deprecated Scripts"]
RemoveDeprecated --> OptimizeKnip["Optimize Knip Configuration"]
OptimizeKnip --> UpdateESLint["Update ESLint Flat Config"]
UpdateESLint --> EnhanceWiki["Enhance Wiki Sync"]
EnhanceWiki --> BuildComplete(["Build Complete"])
Loading

Diagram sources

Section sources

Infrastructure Provisioning and Snapshots

  • Wait-for-infra script ensures external services are ready before running integration tests.
  • Snapshot import and seed scripts prepare deterministic datasets for reproducible tests.
  • Environment creation helper sets up required variables and files.
sequenceDiagram
participant CI as "CI Job"
participant WAIT as "Wait-for-Infra Script"
participant SEED as "Seed/Import Snapshots"
participant SUITES as "Integration Suites"
CI->>WAIT : "Check service readiness"
WAIT-->>CI : "Ready"
CI->>SEED : "Import/seed test data"
SEED-->>CI : "Data prepared"
CI->>SUITES : "Run integration tests"
SUITES-->>CI : "Results"
Loading

Diagram sources

Section sources

Helm Chart Testing

  • Dedicated script runs chart linting and validation.
  • Results are included in the CI summary.
flowchart TD
HStart(["Start Helm Tests"]) --> Lint["Lint Charts"]
Lint --> Validate["Validate Values and Templates"]
Validate --> Report["Report Results"]
Report --> HEnd(["Done"])
Loading

Diagram sources

Section sources

Build Helpers and UI Environment

  • UI environment definition script injects runtime variables during build.
  • Embedded docs build script prepares documentation assets consumed at runtime.
flowchart TD
BStart(["Start Build Helpers"]) --> EnvDef["Define UI Env Variables"]
EnvDef --> DocsBuild["Build Embedded Docs"]
DocsBuild --> BEnd(["Build Complete"])
Loading

Diagram sources

Section sources

Container Images and Entrypoints

  • Multi-stage Dockerfiles for production, development, and stdio modes.
  • Stdio entrypoint script configures runtime behavior for CLI usage.
classDiagram
class Dockerfile {
+multi_stage_build()
+production_image()
}
class Dockerfile_dev {
+dev_dependencies()
+hot_reload_support()
}
class Dockerfile_stdio {
+stdio_runtime()
}
class EntryStdio {
+configure_env()
+start_server()
}
Dockerfile <.. Dockerfile_dev : "shared base"
Dockerfile <.. Dockerfile_stdio : "shared base"
Dockerfile_stdio --> EntryStdio : "uses"
Loading

Diagram sources

Section sources

Pre-commit Hooks and Local Development Automation

  • Husky pre-commit hook enforces code quality and formatting before commits.
  • Can be extended to include additional linters or tests.
flowchart TD
PStart(["Pre-commit Hook"]) --> Lint["Run Linters"]
Lint --> Format["Format Code"]
Format --> QuickTest["Run Quick Tests"]
QuickTest --> PEnd(["Commit Allowed"])
Loading

Diagram sources

Section sources

Security Scanning and Compliance Checks

  • Trivy scans containers and filesystems for vulnerabilities; ignores are managed via an ignore file.
  • CodeQL analysis identifies security issues in source code.
  • Results are published to the workflow summary.
flowchart TD
SStart(["Security Scans"]) --> Trivy["Trivy Scan"]
Trivy --> IgnoreRules["Apply Ignore Rules"]
IgnoreRules --> CodeQL["CodeQL Analysis"]
CodeQL --> Publish["Publish Findings"]
Publish --> SEnd(["Done"])
Loading

Diagram sources

Section sources

Dependency Updates

  • Dependabot monitors package manifests and opens update PRs automatically.
flowchart TD
DStart(["Dependabot Monitor"]) --> Check["Check Manifests"]
Check --> PR["Open Update PR"]
PR --> DEnd(["Review and Merge"])
Loading

Diagram sources

Section sources

Dependency Analysis

The CI workflow depends on:

  • Node.js toolchain and package manager for installs and builds.
  • Test runners configured via Jest and Vitest.
  • External services provisioned by Compose or CI-hosted services.
  • Helm CLI for chart validation.
  • Security scanners (Trivy, CodeQL).
  • Updated configuration tools (Knip, ESLint flat config).
graph TB
PKG["package.json"]
JEST["jest.config.js"]
VITEST["vitest.config.ts"]
SEQ["tests/jest-sequencer.cjs"]
RPT["tests/reporters/jest-github-summary-reporter.cjs"]
COMPOSE["compose.yaml"]
HELM["scripts/test-helm.sh"]
TRIVY[".trivyignore"]
CODEQL[".github/workflows/codeql-analysis.yml"]
INTEGRATION[".github/workflows/integration.yml"]
KNIP["knip.config.ts"]
ESLINT["eslint/flat-config.cjs"]
PKG --> JEST
PKG --> VITEST
JEST --> SEQ
JEST --> RPT
COMPOSE --> JEST
HELM --> JEST
TRIVY --> CODEQL
INTEGRATION --> JEST
INTEGRATION --> RPT
KNIP --> INTEGRATION
ESLINT --> INTEGRATION
Loading

Diagram sources

Section sources

Performance Considerations

Updated Performance optimizations have been enhanced through workflow consolidation, two-phase testing strategy, and streamlined build configuration.

  • Caching:
    • Restore and save Node modules and build caches between jobs to minimize install times.
    • Cache test snapshots where appropriate to speed up integration tests.
    • New: Optimized cache sharing between Phase 1 and Phase 2 tests to reduce redundant setup.
  • Parallelization:
    • Use matrix strategies to split suites across workers.
    • Leverage the parallel checks script to distribute workloads efficiently.
    • New: Phase-aware worker allocation optimizes resource usage based on test type.
  • Artifact reuse:
    • Upload build artifacts once and consume them in subsequent jobs to avoid redundant builds.
    • New: Consolidated artifact management reduces upload/download overhead.
  • Concurrency limits:
    • Configure runner concurrency to prevent resource contention.
    • New: Dynamic concurrency adjustment based on test phase requirements.
  • Image optimization:
    • Use multi-stage Dockerfiles to keep images lean and reduce scan times.
  • Two-Phase Optimization:
    • New: Fail-fast behavior eliminates unnecessary Phase 2 execution when Phase 1 fails.
    • New: Read-only tests execute faster, providing quicker feedback.
    • New: Reduced overall workflow duration through intelligent test ordering.
  • Build Configuration Optimization:
    • New: Removed deprecated npm scripts reduce build complexity and improve performance.
    • New: Enhanced wiki synchronization minimizes overhead in CI pipelines.
    • New: Optimized Knip and ESLint configurations provide faster analysis.

Troubleshooting Guide

Updated Enhanced troubleshooting guidance for the consolidated workflow, two-phase testing strategy, and streamlined build configuration.

  • Debugging CI failures:
    • Inspect step summaries generated by the summary script for aggregated results.
    • Download artifacts containing logs and reports for deeper analysis.
    • Use wait-for-infra logs to verify external service readiness.
    • New: Check Phase 1 vs Phase 2 failure indicators in workflow output.
    • New: Review consolidated workflow logs instead of separate integration workflow logs.
    • New: Verify build configuration changes don't break existing workflows.
  • Common issues:
    • Missing environment variables: Ensure create-env and deploy-run-env scripts are executed in the correct order.
    • Snapshot mismatches: Re-seed or import snapshots if test data drift occurs.
    • Helm validation errors: Review values and templates referenced by the Helm test script.
    • Security findings: Adjust .trivyignore only when justified; otherwise remediate vulnerabilities.
    • New: Phase 1 failures preventing Phase 2 execution - verify read-only test dependencies.
    • New: Resource contention in unified workflow - adjust matrix configuration if needed.
    • New: Build configuration issues - check for removed npm scripts and updated configuration files.
  • Optimization tips:
    • Increase cache keys specificity to avoid stale caches.
    • Reduce suite size or shard further if tests exceed timeouts.
    • Pin Node.js versions to ensure consistent builds.
    • New: Optimize test partitioning between phases for balanced execution time.
    • New: Monitor Phase 1 completion time to identify slow read-only tests.
    • New: Leverage streamlined build configuration for faster CI execution.

Section sources

Conclusion

Kairos MCP's CI system has been significantly streamlined through consolidation of three separate integration workflows into a single unified integration.yml workflow, implementing a new two-phase testing strategy with read-only-first fail-fast behavior. Recent enhancements have further optimized the build configuration by removing deprecated npm scripts and improving GitHub Actions workflows for wiki synchronization. These improvements deliver faster feedback loops, improved resource utilization, simplified maintenance, and reduced build complexity while maintaining comprehensive test coverage. The modular scripts and clear separation of concerns make it straightforward to extend pipelines, add new checks, and optimize performance. Adopting the recommended practices will help maintain high-quality releases and secure deployments with enhanced efficiency.

Appendices

Example Custom CI Scripts and Tasks

  • Unified workflow orchestration: Consolidates multiple integration workflows into single streamlined pipeline.
  • Two-phase testing: Implements read-only-first fail-fast strategy for faster feedback.
  • Parallel checks: Distribute test suites across workers for faster execution with phase-aware distribution.
  • Step summary: Aggregate results into a single GitHub summary for visibility.
  • TGZ install test: Validate packaged artifacts installation flows.
  • Infra wait: Poll external services until healthy before running dependent tests.
  • Helm tests: Lint and validate charts consistently across environments.
  • Snapshot management: Import and seed deterministic datasets for stable tests.
  • Build helpers: Define UI env variables and build embedded docs for runtime consumption.
  • Deploy environment: Prepare runtime environment variables and secrets for deployment jobs.
  • Stdio entrypoint: Configure CLI runtime behavior for headless operations.
  • Streamlined build configuration: Optimized npm scripts with enhanced wiki synchronization.
  • Configuration management: Updated Knip and ESLint configurations for improved code analysis.

Section sources

KAIROS MCP

Clone this wiki locally