Enhancement: Sprint 1: OAI-locked

[clduab11]

🚀 Sprint 1 Implementation: OpenAI-Only Platform Foundation

📋 Summary

Implement the foundational Sprint 1 deliverables from coordination/openai_only_platform_plan.json to transition the OpenAI-only roadmap from planning to execution. This sprint establishes core instruction processing, routing capabilities, and persistence infrastructure required for the entire platform.

🎯 Sprint 1 Objectives

• Primary Goal: Native instruction processing with OpenAI endpoints
• Secondary Goal: Intelligent routing with persona-aligned decision making
• Infrastructure Goal: Persistent memory foundation for all future sprints

🔧 Technical Tasks

Core Infrastructure

• Instruction Parser Module (src/instructions/parser.ts)

  • Implement recursive AGENT.md discovery across repository structure
  • Build precedence handling system (global → project → local → override hierarchy)
  • Create SQLite-backed caching mechanism (memory/instructions.db)
  • Support metadata extraction and validation
  • Add error handling for malformed instruction files
  • Complexity: High

• CLI Integration (src/cli/index.ts)

  • Wire codex-synaptic run --codex command with instruction streaming
  • Implement codex-synaptic instructions sync for cache management
  • Add codex-synaptic instructions validate for syntax checking
  • Create progress indicators and verbose logging options
  • Author comprehensive documentation (docs/cli/instructions.md)
  • Complexity: Medium |

• Routing Policy Service (API endpoints)

  • Implement POST /v1/router/evaluate with persona-aligned embeddings
  • Build POST /v1/router/rules for dynamic policy management
  • Create configuration system (config/routing/policies.json)
  • Add request/response logging and audit trails
  • Implement fallback mechanisms for routing failures
  • Complexity: High

Persistence & Memory

• Storage Infrastructure
  • Set up memory/instructions.db (SQLite schema with indexes)
  • Create memory/routing/history.parquet for evaluation storage
  • Implement backup rotation policy (7-day retention)
  • Add database migration system for schema evolution
  • Create health check endpoints for storage systems
  • Complexity: Medium

Testing & Quality Assurance

• Comprehensive Test Suite
  • Unit tests: tests/instructions/parser.spec.ts (>90% coverage)
  • Integration tests: tests/router/rules.spec.ts (API contract testing)
  • End-to-end tests: tests/cli/instructions.e2e.ts (full workflow)
  • Performance tests for instruction parsing at scale
  • Mock external dependencies (OpenAI API calls)
  • Complexity: Medium

📊 Acceptance Criteria

Functional Requirements

• Instruction parser successfully processes nested AGENT.md files with 100% accuracy
• CLI commands execute without errors and provide meaningful feedback
• Routing APIs return consistent results within 500ms response time
• All persistent storage artifacts follow repository backup policies
• System handles graceful degradation when OpenAI endpoints are unavailable

Technical Requirements

• All new modules achieve >85% test coverage
• TypeScript compilation passes with strict mode enabled
• Vitest test suite completes in <30 seconds
• Memory usage remains <100MB during normal operation
• All APIs follow OpenAPI 3.0 specification standards

Documentation Requirements

• CLI documentation includes usage examples and troubleshooting
• API endpoints documented with request/response schemas
• Architecture decision records (ADRs) created for major design choices
• Integration guide covers local development setup

🔗 Dependencies & Prerequisites

Technical Dependencies

• Node.js 18+ with TypeScript 5.0+
• SQLite3 development libraries
• Apache Parquet support (via DuckDB integration)
• OpenAI API access with sufficient quota

Implementation Order

1. Persistence infrastructure (foundation)
2. Instruction parser (core functionality)
3. CLI integration (user interface)
4. Routing service (advanced features)
5. Comprehensive testing (quality assurance)

⚠️ Risk Mitigation

High-Risk Areas

• OpenAI API Rate Limits: Implement exponential backoff and circuit breaker patterns
• Large Repository Processing: Add streaming and chunked processing for scalability
• Database Corruption: Implement atomic transactions and backup validation
• Memory Leaks: Monitor and profile memory usage during development

Technical Debt Prevention

• Use dependency injection for better testability
• Implement configuration validation at startup
• Add structured logging with correlation IDs
• Create performance benchmarks for regression detection

📈 Success Metrics

• Instruction parsing processes 1000+ files in <10 seconds
• Routing decisions maintain >95% consistency across requests
• CLI commands provide clear error messages for all failure modes
• System startup time remains under 3 seconds
• Zero data loss during normal operation and graceful shutdowns

🔍 Implementation Notes

• Follow the persona alignment heuristics defined in the planning document
• Ensure all storage operations are atomic and transactional
• Implement comprehensive logging for debugging and monitoring
• Use feature flags for gradual rollout of new capabilities
• Consider future Sprint 2-4 requirements during architectural decisions

📚 References

• Planning Document: coordination/openai_only_platform_plan.json (Lines 51-152)
• Architecture Patterns: Capability blueprints section
• Testing Strategy: Lines 300-320 in planning document
• Risk Register: Global and Sprint 1 specific mitigations

[traycerai]

Plan

Observations

Based on my comprehensive exploration of the codebase, I have a clear understanding of the current architecture and Sprint 1 requirements. The project is a sophisticated TypeScript-based distributed agent system with:

• Existing Infrastructure: Well-structured CLI, agent registry, task scheduler, in-memory storage, and SQLite-based memory system
• Missing Components: No HTTP server, no instruction processing, no routing capabilities, no SQLite-backed instruction caching
• Integration Points: Clear patterns for extending storage, CLI commands, and task execution flows
• Sprint 1 Scope: Add instruction parser, CLI integration, routing APIs, and comprehensive testing

The planning document provides detailed specifications, and the current codebase has solid foundations to build upon without major architectural changes.

Approach

High-Level Strategy: Implement Sprint 1 as a foundational layer that extends the existing architecture without breaking current functionality. Follow the mandated implementation order: storage → parser → CLI → router → testing.

Key Design Decisions:

• Extend existing storage patterns rather than replacing them
• Add HTTP server as a new service alongside the existing system
• Integrate instruction commands into the existing CLI framework
• Follow established patterns for logging, configuration, and error handling
• Maintain backward compatibility with all existing functionality

Architecture Integration:

• New instruction parser will integrate with the existing agent system
• Router service will be a standalone HTTP service that can be started alongside the main system
• CLI extensions will follow existing command patterns and use the same system integration points
• Storage extensions will coexist with current memory systems

Reasoning

I systematically explored the codebase to understand the current architecture and Sprint 1 requirements. I examined the planning document to understand the detailed specifications, analyzed the existing CLI structure and core types, investigated the current storage and memory systems, and used specialized agents to explore HTTP infrastructure, task execution flows, and memory system details. This comprehensive analysis revealed the integration points and patterns needed for Sprint 1 implementation.

Proposed File Changes

📁 memory/routing ^(NEW) 🔗

Create subdirectory for routing-related persistent storage files, specifically for storing routing decision history in Parquet format. This follows the planning document's specification for memory/routing/history.parquet storage layout.

📁 config/instructions ^(NEW) 🔗

Create configuration directory for instruction-related settings including scope precedence rules, cache TTL settings, and validation rules. This follows the planning document's specification for config/instructions/*.json configuration artifacts.

📁 config/routing ^(NEW) 🔗

Create configuration directory for routing policies, persona weights, and routing decision rules. This will house the policies.json file specified in the planning document and support the router service configuration needs.

📁 src/instructions ^(NEW) 🔗

Create the main instructions module directory that will contain the instruction parser, validation logic, and related utilities. This is a core Sprint 1 deliverable as specified in the planning document.

📁 src/router ^(NEW) 🔗

Create the router module directory for routing policy engine, evaluation logic, and HTTP endpoints. This will house the routing service that provides the POST /v1/router/evaluate and POST /v1/router/rules endpoints specified in Sprint 1 requirements.

📁 src/server ^(NEW) 🔗

Create HTTP server module directory since the current codebase has no HTTP infrastructure. This will house the Fastify-based HTTP server that exposes the router API endpoints required by Sprint 1.

📁 src/openai ^(NEW) 🔗

Create OpenAI integration module directory for the OpenAI client wrapper, embedding utilities, and API interaction patterns. This is needed for the persona-aligned embeddings functionality in the router service.

📁 docs/cli ^(NEW) 🔗

Create CLI documentation directory to house the instructions.md documentation file specified in Sprint 1 deliverables. This will provide comprehensive documentation for the new instruction-related CLI commands.

📁 tests/instructions ^(NEW) 🔗

Create test directory for instruction parser unit tests. This will contain parser.spec.ts and related test files to achieve the >90% coverage requirement specified in Sprint 1 acceptance criteria.

📁 tests/router ^(NEW) 🔗

Create test directory for router service unit tests. This will contain rules.spec.ts and API contract testing files to ensure routing functionality meets Sprint 1 requirements.

📄 config/instructions/scopes.json ^(NEW) 🔗

References

• 📄 coordination/openai_only_platform_plan.json

Create instruction scope configuration file defining precedence rules (global → project → local → override hierarchy), cache TTL settings, and validation rules. This configuration will be used by the instruction parser to determine how to handle conflicting AGENT.md files and manage cache behavior. Include settings for cache expiration, precedence weights, and validation strictness levels.

📄 config/routing/policies.json ^(NEW) 🔗

References

• 📄 coordination/openai_only_platform_plan.json

Create routing policy configuration file with persona weights, routing decision rules, and fallback mechanisms. This file will define how the router service evaluates requests and makes routing decisions based on persona alignment. Include configuration for embedding models, similarity thresholds, fallback strategies, and policy update mechanisms.

📄 package.json ^(MODIFY) 🔗

Add new dependencies required for Sprint 1 implementation. Add fastify and @fastify/cors for HTTP server functionality, openai for OpenAI API integration, parquetjs or duckdb for Parquet file handling, fast-glob for file discovery, and gray-matter for AGENT.md front-matter parsing. Update the existing SQLite3 dependency if needed and add any additional TypeScript type definitions required for the new dependencies.

📄 src/core/config.ts ^(MODIFY) 🔗

References

• 📄 config/system.json

Extend the existing configuration system to support instruction and routing configuration sections. Add configuration interfaces for InstructionConfig (cache TTL, precedence rules, validation settings) and RoutingConfig (policies path, fallback strategy, embedding settings). Update the main configuration loader to include these new sections and provide default values. Ensure the configuration system can load from the new config files in config/instructions/ and config/routing/ directories.

📄 src/core/storage.ts ^(MODIFY) 🔗

References

• 📄 src/memory/memory-system.ts

Extend the existing storage system to support pluggable storage backends while maintaining backward compatibility. Add a IStorage interface that defines the contract for storage implementations (initialize(), shutdown(), set(), get(), getMetrics()). Modify StorageManager to accept optional storage backend injection while defaulting to MemoryStorage for existing functionality. Add a new SqliteInstructionStore class that implements the interface and provides SQLite-backed storage for instruction caching. Ensure the storage manager can route different namespaces to different backends (e.g., 'instructions' namespace to SQLite, others to memory).

📄 src/instructions/types.ts ^(NEW) 🔗

References

• 📄 src/core/types.ts

Define TypeScript interfaces and types for the instruction system. Create interfaces for ParsedInstruction (path, scope, checksum, metadata, content), InstructionScope enum (global, project, local, override), ValidationResult (isValid, errors, warnings), DiscoverOptions (include patterns, exclude patterns, max depth), and InstructionMetadata (author, version, capabilities, dependencies). These types will be used throughout the instruction parser and related modules to ensure type safety and consistency.

📄 src/instructions/parser.ts ^(NEW) 🔗

References

• 📄 src/core/storage.ts ^(MODIFY)
• 📄 src/core/logger.ts

Implement the core instruction parser module as specified in Sprint 1 requirements. Create functions for discover() to recursively find AGENT.md files across repository structure, parseInstruction() to extract metadata and content from individual files, resolvePrecedence() to handle the global → project → local → override hierarchy, and validateInstruction() to check for malformed files. Implement SQLite-backed caching using the extended storage system from src/core/storage.ts. Include error handling for malformed instruction files, checksum calculation for cache invalidation, and metadata extraction from front-matter. The parser should integrate with the existing logging system and follow established error handling patterns.

📄 src/instructions/cache.ts ^(NEW) 🔗

References

• 📄 src/memory/memory-system.ts
• 📄 src/core/storage.ts ^(MODIFY)

Implement SQLite-backed instruction caching system. Create InstructionCache class that manages the memory/instructions.db SQLite database with tables for storing parsed instructions, metadata, and cache metadata. Implement methods for put(), get(), invalidate(), clear(), and getStats(). Include cache TTL management, checksum-based invalidation, and atomic transaction support. Follow the existing SQLite patterns established in src/memory/memory-system.ts but create a separate database file specifically for instruction caching. Ensure proper error handling and database migration support for schema evolution.

📄 src/instructions/validator.ts ^(NEW) 🔗

References

• 📄 src/instructions/types.ts ^(NEW)

Implement instruction validation logic for checking AGENT.md file syntax, metadata completeness, and content structure. Create validation functions for front-matter format, required fields, capability declarations, and dependency specifications. Implement schema validation for instruction metadata and content structure. Provide detailed error messages and warnings for malformed files, missing required fields, and potential conflicts. Include validation rules for instruction precedence conflicts and circular dependencies. The validator should integrate with the parser and provide actionable feedback for the CLI validation command.

📄 src/instructions/index.ts ^(NEW) 🔗

References

• 📄 src/instructions/parser.ts ^(NEW)
• 📄 src/instructions/cache.ts ^(NEW)
• 📄 src/instructions/validator.ts ^(NEW)

Create the main instruction module export file that exposes the public API for the instruction system. Export key functions from parser, cache, and validator modules. Provide a unified interface for CLI and other consumers to interact with instruction functionality. Include convenience functions that combine parsing, caching, and validation operations. This file serves as the main entry point for instruction-related functionality and should provide a clean, well-documented API surface.

📄 src/openai/client.ts ^(NEW) 🔗

References

• 📄 src/core/config.ts ^(MODIFY)
• 📄 src/core/logger.ts

Implement OpenAI API client wrapper with embedding functionality for persona-aligned routing. Create OpenAIClient class that handles API authentication, rate limiting, and error handling. Implement methods for generating embeddings from text content, with support for different embedding models. Include exponential backoff and circuit breaker patterns for handling API rate limits and failures. Provide caching for embeddings to reduce API calls. The client should integrate with the existing configuration system for API keys and settings, and follow established logging and error handling patterns.

📄 src/openai/embeddings.ts ^(NEW) 🔗

References

• 📄 src/openai/client.ts ^(NEW)

Implement embedding utilities for persona alignment and similarity calculations. Create functions for generating embeddings from instruction content, calculating cosine similarity between embeddings, and managing embedding caches. Implement persona alignment logic that compares request embeddings with stored persona profiles. Include utilities for batch embedding generation and similarity threshold evaluation. The module should provide efficient embedding operations for the router service while minimizing OpenAI API calls through intelligent caching.

📄 src/openai/index.ts ^(NEW) 🔗

References

• 📄 src/openai/client.ts ^(NEW)
• 📄 src/openai/embeddings.ts ^(NEW)

Create the main OpenAI module export file that provides a unified interface for OpenAI integration functionality. Export the client, embedding utilities, and any helper functions. Provide factory functions for creating configured client instances and embedding services. This module serves as the integration point between the router service and OpenAI API functionality.

📄 src/router/types.ts ^(NEW) 🔗

References

• 📄 src/core/types.ts

Define TypeScript interfaces for the routing system. Create interfaces for RoutingRequest (content, context, metadata), RoutingResponse (selectedAgent, confidence, reasoning), RoutingPolicy (rules, weights, thresholds), PersonaProfile (name, description, embeddings, capabilities), and RoutingHistory (request, response, timestamp, performance). These types will ensure type safety across the routing service and provide clear contracts for API endpoints.

📄 src/router/policy-engine.ts ^(NEW) 🔗

References

• 📄 src/openai/embeddings.ts ^(NEW)
• 📄 src/router/types.ts ^(NEW)

Implement the core routing policy engine that evaluates requests and makes routing decisions. Create PolicyEngine class with methods for evaluate() to process routing requests, loadPolicies() to read configuration, and updateRules() to modify routing behavior. Implement persona-aligned decision making using embeddings from the OpenAI module. Include fallback mechanisms for routing failures, policy caching for performance, and audit logging for routing decisions. The engine should integrate with the Parquet storage for persisting routing history and provide consistent results within the 500ms response time requirement.

📄 src/router/history.ts ^(NEW) 🔗

References

• 📄 src/router/types.ts ^(NEW)

Implement Parquet-based storage for routing decision history. Create RoutingHistory class that manages the memory/routing/history.parquet file for storing evaluation results, performance metrics, and decision audit trails. Implement methods for append() to add new routing decisions, query() to retrieve historical data, and analyze() to generate routing performance insights. Include batch writing for performance, schema management for data evolution, and compression for storage efficiency. The history system should support A/B testing analysis and routing optimization feedback loops.

📄 src/router/index.ts ^(NEW) 🔗

References

• 📄 src/router/policy-engine.ts ^(NEW)
• 📄 src/router/history.ts ^(NEW)

Create the main router module export file that provides the public API for routing functionality. Export the policy engine, history management, and routing utilities. Provide factory functions for creating configured router instances and convenience methods for common routing operations. This module serves as the integration point between the HTTP server and routing logic.

📄 src/server/routes.ts ^(NEW) 🔗

References

• 📄 src/router/policy-engine.ts ^(NEW)
• 📄 src/router/types.ts ^(NEW)

Implement HTTP route handlers for the router API endpoints. Create route handlers for POST /v1/router/evaluate that accepts routing requests and returns routing decisions, and POST /v1/router/rules that allows dynamic policy management. Implement request validation, error handling, and response formatting. Include OpenAPI schema definitions for request/response validation. The routes should integrate with the policy engine from the router module and provide proper HTTP status codes and error messages. Ensure all endpoints meet the 500ms response time requirement through efficient implementation.

📄 src/server/index.ts ^(NEW) 🔗

References

• 📄 src/server/routes.ts ^(NEW)
• 📄 src/core/config.ts ^(MODIFY)
• 📄 src/core/logger.ts

Implement the HTTP server using Fastify framework. Create RouterServer class that initializes the HTTP server, registers routes, and manages server lifecycle. Include middleware for CORS, request logging, error handling, and request validation. Implement health check endpoints and metrics collection. The server should be configurable through the existing configuration system and integrate with the existing logging infrastructure. Provide methods for starting and stopping the server, and ensure graceful shutdown handling.

📄 src/cli/instructions.ts ^(NEW) 🔗

References

• 📄 src/cli/index.ts ^(MODIFY)
• 📄 src/instructions/index.ts ^(NEW)

Implement CLI command handlers for instruction-related operations. Create command handlers for instructions sync (cache management), instructions validate (syntax checking), and the --codex flag for the run command. Implement progress indicators, verbose logging options, and comprehensive error handling. The commands should integrate with the instruction parser and provide meaningful feedback to users. Include support for batch operations and parallel processing where appropriate. Follow the existing CLI patterns established in src/cli/index.ts for consistency.

📄 src/cli/index.ts ^(MODIFY) 🔗

References

• 📄 src/cli/instructions.ts ^(NEW)
• 📄 src/cli/codex-context.ts

Integrate the new instruction-related CLI commands into the existing CLI framework. Add command registration for codex-synaptic instructions sync, codex-synaptic instructions validate, and modify the existing task execution commands to support the --codex flag. Import and integrate the instruction command handlers from src/cli/instructions.ts. Ensure the new commands follow the existing CLI patterns for error handling, progress reporting, and user feedback. Update the CLI help text and command structure to include the new instruction functionality while maintaining backward compatibility with existing commands.

📄 src/core/system.ts ^(MODIFY) 🔗

References

• 📄 src/instructions/index.ts ^(NEW)
• 📄 src/router/index.ts ^(NEW)
• 📄 src/server/index.ts ^(NEW)

Integrate the new instruction and router services into the existing system architecture. Add initialization and management of the instruction parser, router service, and HTTP server to the CodexSynapticSystem class. Modify the system startup sequence to initialize the new services and ensure proper shutdown handling. Add methods for accessing instruction and routing functionality from other system components. Ensure the new services integrate with the existing configuration, logging, and error handling systems. The integration should maintain backward compatibility and not affect existing system functionality.

📄 docs/cli/instructions.md ^(NEW) 🔗

References

• 📄 src/cli/instructions.ts ^(NEW)
• 📄 coordination/openai_only_platform_plan.json

Create comprehensive CLI documentation for instruction-related commands as specified in Sprint 1 deliverables. Document the usage, options, and examples for codex-synaptic instructions sync, codex-synaptic instructions validate, and the --codex flag for run commands. Include troubleshooting guides, common error scenarios, and best practices for instruction management. Provide examples of AGENT.md file structure, precedence rules, and validation requirements. The documentation should be clear, comprehensive, and include practical examples that help users understand and effectively use the instruction system.

📄 tests/instructions/parser.spec.ts ^(NEW) 🔗

References

• 📄 src/instructions/parser.ts ^(NEW)
• 📄 tests/agents/registry.test.ts

Implement comprehensive unit tests for the instruction parser module to achieve >90% test coverage as specified in Sprint 1 acceptance criteria. Create test cases for file discovery, parsing logic, precedence resolution, cache operations, and error handling. Include tests for malformed AGENT.md files, missing metadata, circular dependencies, and edge cases. Use mock file systems and temporary directories for testing. Ensure tests run efficiently and complete within the 30-second test suite requirement. Follow existing testing patterns established in the current test suite.

📄 tests/router/rules.spec.ts ^(NEW) 🔗

References

• 📄 src/router/policy-engine.ts ^(NEW)
• 📄 src/server/routes.ts ^(NEW)

Implement unit tests for the router policy engine and API contract testing. Create test cases for routing decision logic, policy evaluation, persona alignment, and API endpoint behavior. Include tests for edge cases, fallback mechanisms, and performance requirements. Mock OpenAI API calls and embedding operations for consistent testing. Test the HTTP endpoints with various request scenarios and validate response formats. Ensure tests verify the 500ms response time requirement and routing consistency. Follow established testing patterns and achieve comprehensive coverage of routing functionality.

📄 tests/cli/instructions.e2e.ts ^(NEW) 🔗

References

• 📄 src/cli/instructions.ts ^(NEW)
• 📄 src/instructions/index.ts ^(NEW)
• 📄 tests/cli/session.test.ts

Implement end-to-end tests for the complete instruction workflow including CLI commands, file processing, and system integration. Create test scenarios that cover the full workflow: instruction discovery → parsing → caching → validation → CLI interaction. Test the instructions sync, instructions validate, and run --codex commands with realistic repository structures and AGENT.md files. Include tests for error scenarios, progress reporting, and user feedback. Use temporary test repositories and mock external dependencies. Ensure tests validate the complete integration between CLI, parser, cache, and system components.

Diagram

sequenceDiagram
    participant CLI as CLI Commands
    participant Parser as Instruction Parser
    participant Cache as SQLite Cache
    participant Router as Router Service
    participant OpenAI as OpenAI API
    participant Storage as Parquet Storage

    Note over CLI,Storage: Sprint 1: Instruction Processing & Routing Flow

    CLI->>Parser: codex-synaptic instructions sync
    Parser->>Parser: Discover AGENT.md files
    Parser->>Parser: Apply precedence rules
    Parser->>Cache: Store parsed instructions
    Cache->>Cache: Update SQLite database
    Parser->>CLI: Return sync results

    CLI->>Parser: codex-synaptic instructions validate
    Parser->>Cache: Retrieve cached instructions
    Parser->>Parser: Validate syntax & metadata
    Parser->>CLI: Return validation results

    CLI->>Parser: codex-synaptic run --codex
    Parser->>Cache: Load instruction context
    Parser->>Router: Send routing request
    Router->>OpenAI: Generate embeddings
    OpenAI->>Router: Return embeddings
    Router->>Router: Evaluate persona alignment
    Router->>Storage: Log routing decision
    Router->>CLI: Return routing response
    CLI->>CLI: Execute with context

    Note over Router,Storage: Router API Endpoints
    Router->>Router: POST /v1/router/evaluate
    Router->>Router: POST /v1/router/rules
    Router->>Storage: Persist routing history

Loading

Import In IDE

🤖 Prompt for AI Agents

I have created the following plan after thorough exploration and analysis of the codebase. Follow the below plan verbatim. Trust the files and references. Do not re-verify what's written in the plan. Explore only when absolutely necessary. First implement all the proposed file changes and then I'll review all the changes together at the end.

### Observations

Based on my comprehensive exploration of the codebase, I have a clear understanding of the current architecture and Sprint 1 requirements. The project is a sophisticated TypeScript-based distributed agent system with:

- **Existing Infrastructure**: Well-structured CLI, agent registry, task scheduler, in-memory storage, and SQLite-based memory system
- **Missing Components**: No HTTP server, no instruction processing, no routing capabilities, no SQLite-backed instruction caching
- **Integration Points**: Clear patterns for extending storage, CLI commands, and task execution flows
- **Sprint 1 Scope**: Add instruction parser, CLI integration, routing APIs, and comprehensive testing

The planning document provides detailed specifications, and the current codebase has solid foundations to build upon without major architectural changes.

### Approach

**High-Level Strategy**: Implement Sprint 1 as a foundational layer that extends the existing architecture without breaking current functionality. Follow the mandated implementation order: storage → parser → CLI → router → testing.

**Key Design Decisions**:
- Extend existing storage patterns rather than replacing them
- Add HTTP server as a new service alongside the existing system
- Integrate instruction commands into the existing CLI framework
- Follow established patterns for logging, configuration, and error handling
- Maintain backward compatibility with all existing functionality

**Architecture Integration**:
- New instruction parser will integrate with the existing agent system
- Router service will be a standalone HTTP service that can be started alongside the main system
- CLI extensions will follow existing command patterns and use the same system integration points
- Storage extensions will coexist with current memory systems

### Reasoning

I systematically explored the codebase to understand the current architecture and Sprint 1 requirements. I examined the planning document to understand the detailed specifications, analyzed the existing CLI structure and core types, investigated the current storage and memory systems, and used specialized agents to explore HTTP infrastructure, task execution flows, and memory system details. This comprehensive analysis revealed the integration points and patterns needed for Sprint 1 implementation.

## Mermaid Diagram

sequenceDiagram
    participant CLI as CLI Commands
    participant Parser as Instruction Parser
    participant Cache as SQLite Cache
    participant Router as Router Service
    participant OpenAI as OpenAI API
    participant Storage as Parquet Storage

    Note over CLI,Storage: Sprint 1: Instruction Processing & Routing Flow

    CLI->>Parser: codex-synaptic instructions sync
    Parser->>Parser: Discover AGENT.md files
    Parser->>Parser: Apply precedence rules
    Parser->>Cache: Store parsed instructions
    Cache->>Cache: Update SQLite database
    Parser->>CLI: Return sync results

    CLI->>Parser: codex-synaptic instructions validate
    Parser->>Cache: Retrieve cached instructions
    Parser->>Parser: Validate syntax & metadata
    Parser->>CLI: Return validation results

    CLI->>Parser: codex-synaptic run --codex
    Parser->>Cache: Load instruction context
    Parser->>Router: Send routing request
    Router->>OpenAI: Generate embeddings
    OpenAI->>Router: Return embeddings
    Router->>Router: Evaluate persona alignment
    Router->>Storage: Log routing decision
    Router->>CLI: Return routing response
    CLI->>CLI: Execute with context

    Note over Router,Storage: Router API Endpoints
    Router->>Router: POST /v1/router/evaluate
    Router->>Router: POST /v1/router/rules
    Router->>Storage: Persist routing history

## Proposed File Changes

### memory/routing(NEW)

Create subdirectory for routing-related persistent storage files, specifically for storing routing decision history in Parquet format. This follows the planning document's specification for `memory/routing/history.parquet` storage layout.

### config/instructions(NEW)

Create configuration directory for instruction-related settings including scope precedence rules, cache TTL settings, and validation rules. This follows the planning document's specification for `config/instructions/*.json` configuration artifacts.

### config/routing(NEW)

Create configuration directory for routing policies, persona weights, and routing decision rules. This will house the `policies.json` file specified in the planning document and support the router service configuration needs.

### src/instructions(NEW)

Create the main instructions module directory that will contain the instruction parser, validation logic, and related utilities. This is a core Sprint 1 deliverable as specified in the planning document.

### src/router(NEW)

Create the router module directory for routing policy engine, evaluation logic, and HTTP endpoints. This will house the routing service that provides the `POST /v1/router/evaluate` and `POST /v1/router/rules` endpoints specified in Sprint 1 requirements.

### src/server(NEW)

Create HTTP server module directory since the current codebase has no HTTP infrastructure. This will house the Fastify-based HTTP server that exposes the router API endpoints required by Sprint 1.

### src/openai(NEW)

Create OpenAI integration module directory for the OpenAI client wrapper, embedding utilities, and API interaction patterns. This is needed for the persona-aligned embeddings functionality in the router service.

### docs/cli(NEW)

Create CLI documentation directory to house the `instructions.md` documentation file specified in Sprint 1 deliverables. This will provide comprehensive documentation for the new instruction-related CLI commands.

### tests/instructions(NEW)

Create test directory for instruction parser unit tests. This will contain `parser.spec.ts` and related test files to achieve the >90% coverage requirement specified in Sprint 1 acceptance criteria.

### tests/router(NEW)

Create test directory for router service unit tests. This will contain `rules.spec.ts` and API contract testing files to ensure routing functionality meets Sprint 1 requirements.

### config/instructions/scopes.json(NEW)

References: 

- coordination/openai_only_platform_plan.json

Create instruction scope configuration file defining precedence rules (global → project → local → override hierarchy), cache TTL settings, and validation rules. This configuration will be used by the instruction parser to determine how to handle conflicting AGENT.md files and manage cache behavior. Include settings for cache expiration, precedence weights, and validation strictness levels.

### config/routing/policies.json(NEW)

References: 

- coordination/openai_only_platform_plan.json

Create routing policy configuration file with persona weights, routing decision rules, and fallback mechanisms. This file will define how the router service evaluates requests and makes routing decisions based on persona alignment. Include configuration for embedding models, similarity thresholds, fallback strategies, and policy update mechanisms.

### package.json(MODIFY)

Add new dependencies required for Sprint 1 implementation. Add `fastify` and `@fastify/cors` for HTTP server functionality, `openai` for OpenAI API integration, `parquetjs` or `duckdb` for Parquet file handling, `fast-glob` for file discovery, and `gray-matter` for AGENT.md front-matter parsing. Update the existing SQLite3 dependency if needed and add any additional TypeScript type definitions required for the new dependencies.

### src/core/config.ts(MODIFY)

References: 

- config/system.json

Extend the existing configuration system to support instruction and routing configuration sections. Add configuration interfaces for `InstructionConfig` (cache TTL, precedence rules, validation settings) and `RoutingConfig` (policies path, fallback strategy, embedding settings). Update the main configuration loader to include these new sections and provide default values. Ensure the configuration system can load from the new config files in `config/instructions/` and `config/routing/` directories.

### src/core/storage.ts(MODIFY)

References: 

- src/memory/memory-system.ts

Extend the existing storage system to support pluggable storage backends while maintaining backward compatibility. Add a `IStorage` interface that defines the contract for storage implementations (`initialize()`, `shutdown()`, `set()`, `get()`, `getMetrics()`). Modify `StorageManager` to accept optional storage backend injection while defaulting to `MemoryStorage` for existing functionality. Add a new `SqliteInstructionStore` class that implements the interface and provides SQLite-backed storage for instruction caching. Ensure the storage manager can route different namespaces to different backends (e.g., 'instructions' namespace to SQLite, others to memory).

### src/instructions/types.ts(NEW)

References: 

- src/core/types.ts

Define TypeScript interfaces and types for the instruction system. Create interfaces for `ParsedInstruction` (path, scope, checksum, metadata, content), `InstructionScope` enum (global, project, local, override), `ValidationResult` (isValid, errors, warnings), `DiscoverOptions` (include patterns, exclude patterns, max depth), and `InstructionMetadata` (author, version, capabilities, dependencies). These types will be used throughout the instruction parser and related modules to ensure type safety and consistency.

### src/instructions/parser.ts(NEW)

References: 

- src/core/storage.ts(MODIFY)
- src/core/logger.ts

Implement the core instruction parser module as specified in Sprint 1 requirements. Create functions for `discover()` to recursively find AGENT.md files across repository structure, `parseInstruction()` to extract metadata and content from individual files, `resolvePrecedence()` to handle the global → project → local → override hierarchy, and `validateInstruction()` to check for malformed files. Implement SQLite-backed caching using the extended storage system from `src/core/storage.ts`. Include error handling for malformed instruction files, checksum calculation for cache invalidation, and metadata extraction from front-matter. The parser should integrate with the existing logging system and follow established error handling patterns.

### src/instructions/cache.ts(NEW)

References: 

- src/memory/memory-system.ts
- src/core/storage.ts(MODIFY)

Implement SQLite-backed instruction caching system. Create `InstructionCache` class that manages the `memory/instructions.db` SQLite database with tables for storing parsed instructions, metadata, and cache metadata. Implement methods for `put()`, `get()`, `invalidate()`, `clear()`, and `getStats()`. Include cache TTL management, checksum-based invalidation, and atomic transaction support. Follow the existing SQLite patterns established in `src/memory/memory-system.ts` but create a separate database file specifically for instruction caching. Ensure proper error handling and database migration support for schema evolution.

### src/instructions/validator.ts(NEW)

References: 

- src/instructions/types.ts(NEW)

Implement instruction validation logic for checking AGENT.md file syntax, metadata completeness, and content structure. Create validation functions for front-matter format, required fields, capability declarations, and dependency specifications. Implement schema validation for instruction metadata and content structure. Provide detailed error messages and warnings for malformed files, missing required fields, and potential conflicts. Include validation rules for instruction precedence conflicts and circular dependencies. The validator should integrate with the parser and provide actionable feedback for the CLI validation command.

### src/instructions/index.ts(NEW)

References: 

- src/instructions/parser.ts(NEW)
- src/instructions/cache.ts(NEW)
- src/instructions/validator.ts(NEW)

Create the main instruction module export file that exposes the public API for the instruction system. Export key functions from parser, cache, and validator modules. Provide a unified interface for CLI and other consumers to interact with instruction functionality. Include convenience functions that combine parsing, caching, and validation operations. This file serves as the main entry point for instruction-related functionality and should provide a clean, well-documented API surface.

### src/openai/client.ts(NEW)

References: 

- src/core/config.ts(MODIFY)
- src/core/logger.ts

Implement OpenAI API client wrapper with embedding functionality for persona-aligned routing. Create `OpenAIClient` class that handles API authentication, rate limiting, and error handling. Implement methods for generating embeddings from text content, with support for different embedding models. Include exponential backoff and circuit breaker patterns for handling API rate limits and failures. Provide caching for embeddings to reduce API calls. The client should integrate with the existing configuration system for API keys and settings, and follow established logging and error handling patterns.

### src/openai/embeddings.ts(NEW)

References: 

- src/openai/client.ts(NEW)

Implement embedding utilities for persona alignment and similarity calculations. Create functions for generating embeddings from instruction content, calculating cosine similarity between embeddings, and managing embedding caches. Implement persona alignment logic that compares request embeddings with stored persona profiles. Include utilities for batch embedding generation and similarity threshold evaluation. The module should provide efficient embedding operations for the router service while minimizing OpenAI API calls through intelligent caching.

### src/openai/index.ts(NEW)

References: 

- src/openai/client.ts(NEW)
- src/openai/embeddings.ts(NEW)

Create the main OpenAI module export file that provides a unified interface for OpenAI integration functionality. Export the client, embedding utilities, and any helper functions. Provide factory functions for creating configured client instances and embedding services. This module serves as the integration point between the router service and OpenAI API functionality.

### src/router/types.ts(NEW)

References: 

- src/core/types.ts

Define TypeScript interfaces for the routing system. Create interfaces for `RoutingRequest` (content, context, metadata), `RoutingResponse` (selectedAgent, confidence, reasoning), `RoutingPolicy` (rules, weights, thresholds), `PersonaProfile` (name, description, embeddings, capabilities), and `RoutingHistory` (request, response, timestamp, performance). These types will ensure type safety across the routing service and provide clear contracts for API endpoints.

### src/router/policy-engine.ts(NEW)

References: 

- src/openai/embeddings.ts(NEW)
- src/router/types.ts(NEW)

Implement the core routing policy engine that evaluates requests and makes routing decisions. Create `PolicyEngine` class with methods for `evaluate()` to process routing requests, `loadPolicies()` to read configuration, and `updateRules()` to modify routing behavior. Implement persona-aligned decision making using embeddings from the OpenAI module. Include fallback mechanisms for routing failures, policy caching for performance, and audit logging for routing decisions. The engine should integrate with the Parquet storage for persisting routing history and provide consistent results within the 500ms response time requirement.

### src/router/history.ts(NEW)

References: 

- src/router/types.ts(NEW)

Implement Parquet-based storage for routing decision history. Create `RoutingHistory` class that manages the `memory/routing/history.parquet` file for storing evaluation results, performance metrics, and decision audit trails. Implement methods for `append()` to add new routing decisions, `query()` to retrieve historical data, and `analyze()` to generate routing performance insights. Include batch writing for performance, schema management for data evolution, and compression for storage efficiency. The history system should support A/B testing analysis and routing optimization feedback loops.

### src/router/index.ts(NEW)

References: 

- src/router/policy-engine.ts(NEW)
- src/router/history.ts(NEW)

Create the main router module export file that provides the public API for routing functionality. Export the policy engine, history management, and routing utilities. Provide factory functions for creating configured router instances and convenience methods for common routing operations. This module serves as the integration point between the HTTP server and routing logic.

### src/server/routes.ts(NEW)

References: 

- src/router/policy-engine.ts(NEW)
- src/router/types.ts(NEW)

Implement HTTP route handlers for the router API endpoints. Create route handlers for `POST /v1/router/evaluate` that accepts routing requests and returns routing decisions, and `POST /v1/router/rules` that allows dynamic policy management. Implement request validation, error handling, and response formatting. Include OpenAPI schema definitions for request/response validation. The routes should integrate with the policy engine from the router module and provide proper HTTP status codes and error messages. Ensure all endpoints meet the 500ms response time requirement through efficient implementation.

### src/server/index.ts(NEW)

References: 

- src/server/routes.ts(NEW)
- src/core/config.ts(MODIFY)
- src/core/logger.ts

Implement the HTTP server using Fastify framework. Create `RouterServer` class that initializes the HTTP server, registers routes, and manages server lifecycle. Include middleware for CORS, request logging, error handling, and request validation. Implement health check endpoints and metrics collection. The server should be configurable through the existing configuration system and integrate with the existing logging infrastructure. Provide methods for starting and stopping the server, and ensure graceful shutdown handling.

### src/cli/instructions.ts(NEW)

References: 

- src/cli/index.ts(MODIFY)
- src/instructions/index.ts(NEW)

Implement CLI command handlers for instruction-related operations. Create command handlers for `instructions sync` (cache management), `instructions validate` (syntax checking), and the `--codex` flag for the `run` command. Implement progress indicators, verbose logging options, and comprehensive error handling. The commands should integrate with the instruction parser and provide meaningful feedback to users. Include support for batch operations and parallel processing where appropriate. Follow the existing CLI patterns established in `src/cli/index.ts` for consistency.

### src/cli/index.ts(MODIFY)

References: 

- src/cli/instructions.ts(NEW)
- src/cli/codex-context.ts

Integrate the new instruction-related CLI commands into the existing CLI framework. Add command registration for `codex-synaptic instructions sync`, `codex-synaptic instructions validate`, and modify the existing task execution commands to support the `--codex` flag. Import and integrate the instruction command handlers from `src/cli/instructions.ts`. Ensure the new commands follow the existing CLI patterns for error handling, progress reporting, and user feedback. Update the CLI help text and command structure to include the new instruction functionality while maintaining backward compatibility with existing commands.

### src/core/system.ts(MODIFY)

References: 

- src/instructions/index.ts(NEW)
- src/router/index.ts(NEW)
- src/server/index.ts(NEW)

Integrate the new instruction and router services into the existing system architecture. Add initialization and management of the instruction parser, router service, and HTTP server to the `CodexSynapticSystem` class. Modify the system startup sequence to initialize the new services and ensure proper shutdown handling. Add methods for accessing instruction and routing functionality from other system components. Ensure the new services integrate with the existing configuration, logging, and error handling systems. The integration should maintain backward compatibility and not affect existing system functionality.

### docs/cli/instructions.md(NEW)

References: 

- src/cli/instructions.ts(NEW)
- coordination/openai_only_platform_plan.json

Create comprehensive CLI documentation for instruction-related commands as specified in Sprint 1 deliverables. Document the usage, options, and examples for `codex-synaptic instructions sync`, `codex-synaptic instructions validate`, and the `--codex` flag for run commands. Include troubleshooting guides, common error scenarios, and best practices for instruction management. Provide examples of AGENT.md file structure, precedence rules, and validation requirements. The documentation should be clear, comprehensive, and include practical examples that help users understand and effectively use the instruction system.

### tests/instructions/parser.spec.ts(NEW)

References: 

- src/instructions/parser.ts(NEW)
- tests/agents/registry.test.ts

Implement comprehensive unit tests for the instruction parser module to achieve >90% test coverage as specified in Sprint 1 acceptance criteria. Create test cases for file discovery, parsing logic, precedence resolution, cache operations, and error handling. Include tests for malformed AGENT.md files, missing metadata, circular dependencies, and edge cases. Use mock file systems and temporary directories for testing. Ensure tests run efficiently and complete within the 30-second test suite requirement. Follow existing testing patterns established in the current test suite.

### tests/router/rules.spec.ts(NEW)

References: 

- src/router/policy-engine.ts(NEW)
- src/server/routes.ts(NEW)

Implement unit tests for the router policy engine and API contract testing. Create test cases for routing decision logic, policy evaluation, persona alignment, and API endpoint behavior. Include tests for edge cases, fallback mechanisms, and performance requirements. Mock OpenAI API calls and embedding operations for consistent testing. Test the HTTP endpoints with various request scenarios and validate response formats. Ensure tests verify the 500ms response time requirement and routing consistency. Follow established testing patterns and achieve comprehensive coverage of routing functionality.

### tests/cli/instructions.e2e.ts(NEW)

References: 

- src/cli/instructions.ts(NEW)
- src/instructions/index.ts(NEW)
- tests/cli/session.test.ts

Implement end-to-end tests for the complete instruction workflow including CLI commands, file processing, and system integration. Create test scenarios that cover the full workflow: instruction discovery → parsing → caching → validation → CLI interaction. Test the `instructions sync`, `instructions validate`, and `run --codex` commands with realistic repository structures and AGENT.md files. Include tests for error scenarios, progress reporting, and user feedback. Use temporary test repositories and mock external dependencies. Ensure tests validate the complete integration between CLI, parser, cache, and system components.

---

Developer Humor

🤖 Sprint 1 is like teaching agents to read their own instruction manual—
First they learn to find AGENT.md files in every corner,
Then cache them faster than a squirrel hoards nuts,
Route requests with the wisdom of a GPS that actually works,
And finally serve APIs smoother than a barista's latte art!
☕️📚🔄

---

Execution Information

Branch: main
Commit: b133289

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.