-
Notifications
You must be signed in to change notification settings - Fork 0
Skills & Protocols
Changes Made
- Updated KAIROS skill version from 4.7.1 to 4.7.2 to reflect incremental improvement to audit system's data completeness
- Enhanced audit system documentation to cover improved data completeness and telemetry capture
- Added comprehensive audit logging capabilities for MCP tool calls and request/response monitoring
- Expanded security documentation with detailed audit event specifications and compliance considerations
- Introduction
- Project Structure
- Core Components
- Architecture Overview
- Detailed Component Analysis
- Dependency Analysis
- Performance Considerations
- Troubleshooting Guide
- Conclusion
- Appendices
This document explains how KAIROS MCP manages skills and protocols, and how to develop, author, validate, export, and distribute them. It covers:
- Skill development and structure for authors
- Protocol authoring with markdown syntax, contract definitions, and validation rules
- Skill bundle creation, metadata derivation, and artifact attachment
- Practical examples for developing skills, authoring protocols, and training
- Distribution, version management, and compatibility considerations
- The relationship between skills, protocols, and the training workflow
- New: Enhanced audit system with improved data completeness for compliance and security monitoring
KAIROS MCP organizes skills and protocols across two primary areas:
- skills/: Human-authored skill packages for installation and use
- src/tools/: Server-side tools implementing training, tuning, exporting, and skill bundling
graph TB
subgraph "Skills (Human-authored)"
S1["skills/README.md"]
S2["skills/SKILLS.md"]
S3["skills/kairos/SKILL.md"]
end
subgraph "Server Tools"
T1["src/tools/train.ts"]
T2["src/tools/tune.ts"]
T3["src/tools/export.ts"]
T4["src/tools/dump.ts"]
end
subgraph "Skill Export Pipeline"
E1["src/tools/export-skill-items.ts"]
E2["src/tools/skill-export/assemble-skill-item.ts"]
E3["src/tools/skill-export/derive-metadata.ts"]
E4["src/tools/skill-export/build-skill-md.ts"]
E5["src/tools/skill-export/artifact-files.ts"]
E6["src/tools/skill-export/zip-bundle.ts"]
end
subgraph "Audit System"
A1["src/http/mcp-audit-emit.ts"]
A2["src/services/embedding/audit.ts"]
A3["src/utils/audit-log-events.ts"]
A4["src/utils/audit-mcp-summary.ts"]
end
S3 --> T1
T1 --> T4
T3 --> E1
E1 --> E2
E2 --> E3
E2 --> E4
E2 --> E5
E1 --> E6
A1 --> A2
A2 --> A3
A3 --> A4
Diagram sources
- skills/README.md:1-68
- skills/SKILLS.md:1-62
- skills/kairos/SKILL.md:1-163
- src/tools/train.ts:1-346
- src/tools/tune.ts:1-58
- src/tools/export.ts:1-315
- src/tools/dump.ts:1-190
- src/tools/export-skill-items.ts:1-56
- src/tools/skill-export/assemble-skill-item.ts:1-80
- src/tools/skill-export/derive-metadata.ts:1-113
- src/tools/skill-export/build-skill-md.ts:1-23
- src/tools/skill-export/artifact-files.ts:1-88
- src/tools/skill-export/zip-bundle.ts:1-67
- src/http/mcp-audit-emit.ts:1-200
- src/services/embedding/audit.ts:1-150
- src/utils/audit-log-events.ts:1-100
- src/utils/audit-mcp-summary.ts:1-80
Section sources
- skills/README.md:1-68
- skills/SKILLS.md:1-62
- skills/kairos/SKILL.md:1-163
- Skills: Human-authored packages installed via the skills CLI, containing a SKILL.md frontmatter and optional assets, scripts, and references.
- Protocols: Structured markdown documents describing adapter workflows with required sections and JSON contract blocks.
- Training and Tuning: Tools to register new adapters and update existing ones from markdown or artifacts.
- Export and Bundling: Tools to export adapters as markdown, training JSONL, or skill bundles (ZIP) with metadata and diagnostics.
- New: Enhanced Audit System: Comprehensive logging and monitoring capabilities for MCP tool calls, request/response telemetry, and compliance reporting with improved data completeness.
Section sources
- skills/README.md:1-68
- skills/SKILLS.md:1-62
- src/tools/train.ts:1-346
- src/tools/tune.ts:1-58
- src/tools/export.ts:1-315
- skills/kairos/SKILL.md:93-147
- src/http/mcp-audit-emit.ts:1-200
The skill and protocol lifecycle spans human-authored content and server-side processing with enhanced audit capabilities:
sequenceDiagram
participant Author as "Skill Author"
participant Repo as "skills/"
participant Server as "KAIROS MCP Server"
participant Audit as "Audit System"
participant Store as "Memory/Qdrant"
participant Export as "Export Tool"
Author->>Repo : Create SKILL.md + references/assets
Author->>Server : Train protocol (markdown or artifact)
Server->>Audit : Log tool call events
Server->>Store : Persist adapter layers and metadata
Author->>Server : Export adapter or training data
Server->>Audit : Capture request/response telemetry
Server->>Export : Build skill items and artifacts
Export->>Audit : Generate audit trail
Export-->>Author : skill_tree/skill_zip with diagnostics
Diagram sources
- skills/SKILLS.md:16-47
- src/tools/train.ts:134-238
- src/tools/export.ts:40-269
- src/tools/export-skill-items.ts:17-55
- src/http/mcp-audit-emit.ts:1-200
- Template structure: Each skill is a directory with a required SKILL.md frontmatter and optional references/, assets/, scripts/.
- Validation: Optional validation via skills-ref aligns with the Agent Skills specification.
- Distribution: Skills are installed globally or per-project via the skills CLI.
flowchart TD
Start(["Author creates skill"]) --> Frontmatter["Add YAML frontmatter<br/>name, description, metadata"]
Frontmatter --> Body["Write markdown body<br/>usage, guidance, references"]
Body --> Assets["Optional: assets/, scripts/, references/"]
Assets --> Validate["Optional: skills-ref validation"]
Validate --> Publish["Publish to skills directory<br/>install via npx skills add"]
Diagram sources
- skills/SKILLS.md:16-47
- skills/README.md:12-67
Section sources
- skills/SKILLS.md:16-62
- skills/README.md:1-68
- Markdown syntax: Protocols are authored as markdown with required sections and JSON contract blocks.
- Contract definitions: Each step includes a JSON block with a type field (shell, mcp, user_input, comment, tensor).
- Validation rules: Structural checks ensure presence of Activation Patterns (first H2), Reward Signal (last H2), at least one JSON contract block, consistent fencing, and allowed contract types.
flowchart TD
PStart(["Protocol Markdown"]) --> Headings["Extract H1/H2 headings<br/>ignore code fences"]
Headings --> Sections["Split by H1 sections"]
Sections --> CheckFirst["First H2 == 'Activation Patterns'?"]
Sections --> CheckLast["Last H2 == 'Reward Signal'?"]
Headings --> Contracts["Find JSON contract blocks"]
Contracts --> FenceCheck["Reject plain<br/>with contract JSON"]
Contracts --> TypeCheck["Validate type in {tensor,shell,mcp,user_input,comment}"]
CheckFirst --> Valid{"All checks pass?"}
CheckLast --> Valid
FenceCheck --> Valid
TypeCheck --> Valid
Valid --> |Yes| Accept["Accept protocol"]
Valid --> |No| Reject["Reject with structured reasons"]
Diagram sources
- src/services/memory/validate-protocol-structure.ts:113-186
Section sources
- src/services/memory/validate-protocol-structure.ts:113-186
Updated Enhanced audit system with improved data completeness for compliance and security monitoring.
KAIROS MCP now features an enhanced audit system designed to capture comprehensive telemetry data for compliance, security monitoring, and operational visibility. The system provides multiple levels of audit detail with improved data completeness.
The audit system captures three primary categories of events:
-
MCP Request Events:
-
mcp_request_start: Captures request initiation before space context resolution -
mcp_request_end: Captures request completion in response finish handler -
mcp_tool_call: Logs individual tool invocation with parameters and responses
-
-
Embedding Events:
-
audit.embedding: Embedding service telemetry for vector operations -
audit.anomaly: Anomaly detection system monitoring
-
-
Security Events:
- Authentication and authorization audit trails
- Access control and permission logging
- Compliance reporting data
The audit system operates at multiple verbosity levels (0-3) with progressively detailed data capture:
- Level 0: Minimal audit data - basic request/response timestamps
- Level 1: Standard audit data - includes tool names, parameters, and basic metadata
- Level 2: Enhanced audit data - adds sanitized request arguments and response summaries
- Level 3: Full audit data - comprehensive telemetry with bounded request/response snapshots
Version 4.7.2 introduces significant improvements to audit data completeness:
- Enhanced Telemetry Capture: Improved collection of tool call parameters, response data, and execution context
- Bounded Data Structures: Safe logging mechanisms prevent sensitive data exposure while maintaining audit utility
- Structured Logging: Consistent audit event formatting for automated processing and compliance reporting
- Compliance Alignment: Audit events designed for regulatory compliance and security monitoring requirements
Section sources
- src/http/mcp-audit-emit.ts:1-200
- src/services/embedding/audit.ts:1-150
- src/utils/audit-log-events.ts:1-100
- src/utils/audit-mcp-summary.ts:1-80
- docs/security/audit-log.md:1-200
Updated Added comprehensive decision framework for determining when to route through KAIROS versus execute directly.
KAIROS provides a systematic decision heuristic to help agents determine when to route through KAIROS versus execute directly. The framework is built around the "training-gap test" that identifies when team-specific conventions are required.
| You are thinking… | Route through KAIROS? |
|---|---|
| "What branch naming convention do they use?" | Yes |
| "Is there a required MR/PR template or description format?" | Yes |
| "Do I need to reference a ticket, and how?" | Yes |
| "Are there post-action steps (pipeline watch, approvals, notifications)?" | Yes |
| "Which environment do I deploy to first?" | Yes |
| "What commit message format does this team expect?" | Yes |
"I just need to run npm test / git status / one clear local command." |
No — execute directly |
| "The user asked me to read a file or explain code." | No — not an action intent |
Before routing through KAIROS, verify these conditions in order:
-
Is the user message an action intent? Look for verbs like build, fix, deploy, write, create, implement, debug, publish, migrate, configure, install, update, edit, refactor, review, audit, test, automate, generate, document, plan, design, or descriptions of a procedure. If no action intent is present, KAIROS routing does not apply.
-
Do you have follow-up questions about team-specific conventions? If yes, route through KAIROS — the adapter answers them. If the action is a single obvious command with no ambiguity, execute directly without routing.
-
Is a KAIROS MCP server connected and authenticated? If not, follow the unavailability procedure below.
-
Has the host environment restricted MCP tool access for this session? If so, respect that restriction.
If conditions 1-3 pass (action intent + team-context questions + server available), KAIROS routing is active for this task.
Route through KAIROS:
- "Create a PR for this fix" — needs branch naming, description template, labels
- "Deploy to staging" — needs environment sequence, approval gates, post-deploy checks
- "Write a commit message" — needs team's conventional-commit format and scope rules
- "Release a new version" — needs semver policy, changelog, tag format, CI triggers
Skip KAIROS (execute directly):
- "Run the tests" →
npm run dev:test— single command, no team-specific form - "Check git status" →
git status— read-only, no conventions involved - "Install dependencies" →
npm install— mechanical, no team policy - "Read src/config/index.ts" — not an action intent at all
Rule of thumb: If the task is a single, self-contained command whose correct form is universally obvious (not team-specific), execute directly. If you have any uncertainty about the team-specific "how" — branch naming, ticket format, deploy sequence, commit conventions — activate KAIROS first. The adapter exists to fill the gap between your training and this team's rules.
Multi-step tasks: Even if each individual step seems obvious, route through KAIROS when the task involves multiple sequential actions (e.g., "fix the tests and push"). The adapter may encode team conventions about step ordering, intermediate validation, or post-action obligations that you'd miss if you just chained obvious commands.
Section sources
- skills/kairos/SKILL.md:93-162
- Assembly: For each adapter URI, the system dumps protocol markdown, normalizes vocabulary, derives metadata (slug, name, description), builds SKILL.md, scans for diagnostics, loads artifacts, and computes checksums.
- Deduplication: Slugs are deduplicated and remapped to avoid collisions.
- Packaging: Files are flattened into ZIP layout with a manifest and optional inline base64 encoding.
sequenceDiagram
participant Client as "Caller"
participant Export as "executeExport"
participant Items as "collectSkillExportItemsForZip"
participant Assemble as "assembleSkillExportItem"
participant Meta as "deriveSkillMetadata"
participant Build as "buildSkillMdFile"
participant Art as "loadArtifactFilesForAdapter"
participant Zip as "zipSkillFiles"
Client->>Export : format=skill_zip
Export->>Items : resolve adapter URIs -> items
loop For each adapter
Items->>Assemble : dump + derive metadata
Assemble->>Meta : compute slug/name/description
Assemble->>Build : generate SKILL.md
Assemble->>Art : attach artifacts
Assemble-->>Items : SkillExportItem
end
Items-->>Export : items + primaryUri
Export->>Zip : flatten + zip
Export-->>Client : download_ref or inline base64
Diagram sources
- src/tools/export.ts:176-264
- src/tools/export-skill-items.ts:17-55
- src/tools/skill-export/assemble-skill-item.ts:32-79
- src/tools/skill-export/derive-metadata.ts:62-99
- src/tools/skill-export/build-skill-md.ts:18-22
- src/tools/skill-export/artifact-files.ts:30-87
- src/tools/skill-export/zip-bundle.ts:39-53
Section sources
- src/tools/export.ts:40-269
- src/tools/export-skill-items.ts:17-55
- src/tools/skill-export/assemble-skill-item.ts:32-79
- src/tools/skill-export/derive-metadata.ts:62-99
- src/tools/skill-export/build-skill-md.ts:18-22
- src/tools/skill-export/artifact-files.ts:30-87
- src/tools/skill-export/zip-bundle.ts:11-67
- Train tool: Validates inputs, resolves content (from input or source adapter), normalizes artifact paths, stores adapter layers, and returns URIs and metadata.
- Tune tool: Updates adapter content via Qdrant service.
- Dump tool: Converts stored layers into markdown with protocol frontmatter and optional contract blocks.
sequenceDiagram
participant Client as "Client"
participant Train as "registerTrainTool"
participant Exec as "executeTrain"
participant Dump as "executeDump"
participant Store as "Memory Store"
Client->>Train : train(input)
Train->>Exec : parse + validate + resolve content
Exec->>Store : persist adapter layers
Exec-->>Client : items with uris and metadata
Client->>Dump : dump(kairos : //layer/{uuid})
Dump-->>Client : markdown with protocol frontmatter
Diagram sources
- src/tools/train.ts:240-345
- src/tools/dump.ts:107-189
Section sources
- src/tools/train.ts:134-238
- src/tools/tune.ts:12-57
- src/tools/dump.ts:107-189
-
Developing a skill
- Create a directory under skills/ with a SKILL.md frontmatter and markdown body.
- Optionally add references/, assets/, scripts/.
- Validate locally with skills-ref if available.
- Install via npx skills add debian777/kairos-mcp.
-
Authoring a protocol
- Write markdown with an H1 title and ordered H2 sections.
- Include Activation Patterns as the first H2 and Reward Signal as the last H2.
- Add JSON contract blocks with allowed types.
- Use the validateProtocolStructure checks to ensure compliance.
-
Training and tuning
- Use the train tool to register a new adapter from markdown or artifact.
- Use the tune tool to update an existing adapter's content.
- Use dump to render a markdown representation of an adapter chain.
-
New: Enhanced Audit System
- Configure audit verbosity levels (0-3) based on compliance requirements
- Monitor MCP tool call events for security and operational insights
- Generate compliance reports using structured audit event data
- Implement bounded data capture to protect sensitive information
-
New: KAIROS Decision Framework
- Use the decision matrix to determine routing: ask "Does this task have team-specific conventions that my training data wouldn't know?"
- Apply the conditions checklist: action intent → team-specific questions → server availability → host restrictions
- Follow the examples: route for complex workflows, skip for simple commands
- Remember the training-gap test: if you're uncertain about team-specific "how," activate KAIROS first
-
Exporting and distributing
- Use export with format=skill_zip to produce a ZIP bundle with a manifest and optional inline base64.
- Use export with format=skill_tree to receive a JSON tree of skill items.
- Use export with format=reward_jsonl, sft_jsonl, or preference_jsonl for training datasets.
Section sources
- skills/SKILLS.md:16-62
- src/services/memory/validate-protocol-structure.ts:113-186
- src/tools/train.ts:240-345
- src/tools/tune.ts:12-57
- src/tools/export.ts:176-264
- skills/kairos/SKILL.md:93-162
- src/http/mcp-audit-emit.ts:1-200
- Protocol versioning: Protocols carry an adapter_version when present; exports include adapter_version for single-adapter selections.
- Slug normalization: Deterministic slugs derived from titles or frontmatter; collisions are handled with auto-suffixes up to a limit.
- Compatibility: Skills specify allowed-tools and metadata.version; clients should respect tool availability and version constraints.
- Updated: Audit system versioning: Audit components now track version 4.7.2 with enhanced data completeness improvements.
Section sources
- src/tools/dump.ts:157-166
- src/utils/protocol-slug.ts:58-91
- skills/kairos/SKILL.md:15-19
- src/http/mcp-audit-emit.ts:1-200
The skill export pipeline composes several modules with clear responsibilities and minimal coupling, now enhanced with audit system integration.
graph LR
Export["export.ts"] --> Items["export-skill-items.ts"]
Items --> Assemble["skill-export/assemble-skill-item.ts"]
Assemble --> Derive["skill-export/derive-metadata.ts"]
Assemble --> Build["skill-export/build-skill-md.ts"]
Assemble --> Art["skill-export/artifact-files.ts"]
Items --> Zip["skill-export/zip-bundle.ts"]
Export --> Audit["Audit Integration"]
Audit --> Emit["mcp-audit-emit.ts"]
Audit --> Events["audit-log-events.ts"]
Audit --> Summary["audit-mcp-summary.ts"]
Diagram sources
- src/tools/export.ts:16-25
- src/tools/export-skill-items.ts:7-12
- src/tools/skill-export/assemble-skill-item.ts:5-12
- src/tools/skill-export/derive-metadata.ts:5-6
- src/tools/skill-export/build-skill-md.ts:5-6
- src/tools/skill-export/artifact-files.ts:5-14
- src/tools/skill-export/zip-bundle.ts:5-8
- src/http/mcp-audit-emit.ts:1-200
- src/utils/audit-log-events.ts:1-100
- src/utils/audit-mcp-summary.ts:1-80
Section sources
- src/tools/export.ts:16-25
- src/tools/export-skill-items.ts:7-12
- src/tools/skill-export/assemble-skill-item.ts:5-12
- src/tools/skill-export/derive-metadata.ts:5-6
- src/tools/skill-export/build-skill-md.ts:5-6
- src/tools/skill-export/artifact-files.ts:5-14
- src/tools/skill-export/zip-bundle.ts:5-8
- src/http/mcp-audit-emit.ts:1-200
- src/utils/audit-log-events.ts:1-100
- src/utils/audit-mcp-summary.ts:1-80
- Export streaming: ZIP archives are streamed to avoid buffering the full archive in memory.
- Compression level: Configurable compression level reduces bundle size during skill_zip delivery.
- Caching: Dump reads leverage a Redis cache for memory resources to reduce latency.
- Validation: Structural validation is lightweight and performed before storage to prevent downstream errors.
- Updated: Audit performance: Enhanced audit system implements efficient data capture with configurable verbosity levels to balance completeness and performance impact.
- Training errors: The train tool normalizes error codes and enriches messages with actionable next actions (e.g., opening the creation flow).
- Duplicate or similar adapters: Errors distinguish between duplicates and near-duplicates; use force_update when appropriate.
- Export diagnostics: The export pipeline scans for advisory risks in Markdown and logs warnings for invalid artifact paths or mismatched hashes.
- Protocol validation failures: Use the validation rules to identify missing sections, incorrect contract types, or mixed fencing.
- New: Enhanced Audit System Issues: Use audit verbosity configuration to adjust logging detail levels. Check audit event capture for tool call failures and verify compliance reporting data integrity.
- New: KAIROS routing issues: Use the conditions checklist to diagnose routing problems. Check server connectivity, authentication status, and host restrictions. Follow the unavailability procedure for MCP server issues.
Section sources
- src/tools/train.ts:56-83
- src/tools/export.ts:266-268
- src/tools/skill-export/scan-diagnostics.ts:15-23
- src/services/memory/validate-protocol-structure.ts:163-186
- skills/kairos/SKILL.md:181-199
- src/http/mcp-audit-emit.ts:1-200
KAIROS MCP provides a robust framework for skills and protocols with enhanced audit capabilities:
- Skills are authored following a standardized structure and validated via the Agent Skills specification.
- Protocols are authored with strict markdown and contract rules to ensure reliable execution.
- Training, tuning, and exporting are integrated into MCP tools with clear error handling and observability.
- Skill bundles streamline distribution with metadata, diagnostics, and artifact attachments.
- New: The enhanced audit system provides comprehensive logging and monitoring with improved data completeness for compliance and security purposes.
- New: The KAIROS decision heuristic provides a systematic framework for determining when to route through KAIROS versus execute directly, based on team-specific conventions and training gaps.
- Priority order for slug/name/description: frontmatter name/slug, memory slug, adapter name, H1 title, label fallback.
- Slug normalization ensures deterministic routing and length limits.
Section sources
- src/tools/skill-export/derive-metadata.ts:62-99
- src/utils/protocol-slug.ts:26-36
- skill_tree: JSON tree of skill items with files and diagnostics.
- skill_zip: ZIP bundle with manifest; supports inline_base64 or download_ref.
- Training JSONL: trace_jsonl, reward_jsonl, sft_jsonl, preference_jsonl for adapter-specific training.
Section sources
- src/tools/export.ts:176-264
Updated Comprehensive audit system configuration and best practices.
Verbosity Levels:
- Level 0: Minimal logging for production environments with low overhead
- Level 1: Standard audit for compliance monitoring and basic troubleshooting
- Level 2: Enhanced logging for detailed debugging and security analysis
- Level 3: Full audit for forensic investigations and comprehensive compliance
Audit Event Categories:
- MCP Tool Calls: Complete tool invocation telemetry with parameter sanitization
- Request/Response: Structured request/response capture with bounded data
- System Events: Infrastructure and operational monitoring data
- Security Events: Authentication, authorization, and access control logs
Production Deployment:
- Use audit level 1 for standard compliance monitoring
- Implement log rotation and retention policies
- Configure centralized audit log aggregation
- Set up alerting for critical security events
Development Environment:
- Enable audit level 2 for detailed debugging
- Use structured logging for easy parsing and analysis
- Implement data masking for sensitive information
- Monitor audit log volume and performance impact
Section sources
- src/http/mcp-audit-emit.ts:1-200
- src/utils/audit-log-events.ts:1-100
- src/utils/audit-mcp-summary.ts:1-80
- docs/security/audit-log.md:1-200
Updated Comprehensive examples demonstrating the decision framework in practice.
Scenario 1: Code Review Process
- Question: "How do we handle code review comments and approvals?"
- Decision: Route through KAIROS — involves team-specific approval gates and notification workflows
- Action: Activate KAIROS to follow the established review procedure
Scenario 2: Daily Standup Command
- Question: "Run the daily standup script"
- Decision: Execute directly — single command with no team-specific form
- Action: Run
./scripts/daily-standup.shwithout routing
Scenario 3: Database Migration
- Question: "Apply the database migration for feature X"
- Decision: Route through KAIROS — involves environment sequence, rollback procedures, and monitoring steps
- Action: Activate KAIROS to follow the migration workflow
Scenario 4: Code Formatting
- Question: "Format all Python files"
- Decision: Execute directly — mechanical task with universal formatting standards
- Action: Run
black .without routing
Application Steps:
- Identify action intent: "Create merge request" ✓
- Team-specific questions: "Branch naming convention?" → Yes
- Server availability: Connected and authenticated ✓
- Host restrictions: None detected ✓
Result: Route through KAIROS using the decision matrix to ensure compliance with team conventions.
Section sources
- skills/kairos/SKILL.md:93-162
- docs/business/case-standardize-commits-and-merge-requests.md:1-66