diff --git a/.claude/AGENTS.md b/.claude/AGENTS.md index 1f9c29f90..2a9b1f7cd 100644 --- a/.claude/AGENTS.md +++ b/.claude/AGENTS.md @@ -1,654 +1,554 @@ # Agent Coordination Guide -Comprehensive guide for the SQLSpec agent system, covering agent responsibilities, workflow patterns, MCP tool usage, and workspace management. +Comprehensive guide for the SQLSpec multi-agent workflow system. All agents (Claude Code, Gemini, Codex, etc.) should follow this workflow. + +## Quick Reference + +- **Active work**: `specs/active/{requirement}/` +- **Archived work**: `specs/archive/{requirement}/` +- **Templates**: `specs/template-spec/` +- **Main standards**: `AGENTS.md` (SQLSpec-specific patterns) + +## Agent Responsibilities + +| Agent | Primary Role | Key Tools | Auto-Invoked By | +|-------|--------------|-----------|-----------------| +| **Planner** | Research & planning | zen.planner, zen.consensus, Context7, WebSearch | User (`/plan`) | +| **Expert** | Implementation & orchestration | zen.debug, zen.thinkdeep, zen.analyze, Context7 | User (`/implement`) | +| **Testing** | Comprehensive test creation | pytest, Bash | Expert (automatic) | +| **Docs & Vision** | Documentation, QA, knowledge capture, cleanup | Sphinx, Bash | Expert (automatic) | + +## Complete Workflow + +``` +User runs: /plan {feature-description} + ↓ +┌─────────────────────────────────────────────────────────────┐ +│ PLANNER AGENT │ +│ • Research (guides, Context7, WebSearch) │ +│ • Use zen.planner for structured planning │ +│ • Get zen.consensus on complex decisions │ +│ • Create workspace: specs/active/{requirement}/ │ +│ • Write: prd.md, tasks.md, research/plan.md, recovery.md │ +└─────────────────────────────────────────────────────────────┘ + ↓ +User runs: /implement + ↓ +┌─────────────────────────────────────────────────────────────┐ +│ EXPERT AGENT │ +│ 1. Read plan (prd.md, tasks.md, research/plan.md) │ +│ 2. Research (guides, Context7) │ +│ 3. Implement following AGENTS.md standards │ +│ 4. Self-test & verify │ +│ 5. ──► Auto-Invoke Testing Agent (subagent) │ +│ ├─► Create unit tests │ +│ ├─► Create integration tests │ +│ ├─► Test edge cases │ +│ └─► Verify coverage & all tests pass │ +│ 6. ──► Auto-Invoke Docs & Vision Agent (subagent) │ +│ ├─► Phase 1: Update documentation │ +│ ├─► Phase 2: Quality gate validation │ +│ ├─► Phase 3: Knowledge capture (AGENTS.md+guides) │ +│ ├─► Phase 4: Re-validate after updates │ +│ ├─► Phase 5: Clean tmp/ and archive │ +│ └─► Generate completion report │ +│ 7. Return complete summary │ +└─────────────────────────────────────────────────────────────┘ +``` + +## Agent-Specific Guidance + +### For Planner Agent + +**Responsibilities:** + +- Research-grounded planning +- Structured planning with zen.planner +- Multi-model consensus for complex decisions +- Workspace creation in `specs/active/` + +**Workflow:** -## Agent Responsibilities Matrix - -| Responsibility | Planner | Expert | Testing | Docs & Vision | -|----------------|---------|--------|---------|---------------| -| **Research** | ✅ Primary | ✅ Implementation details | ✅ Test patterns | ✅ Doc standards | -| **Planning** | ✅ Primary | ❌ | ❌ | ❌ | -| **Implementation** | ❌ | ✅ Primary | ✅ Tests only | ❌ | -| **Testing** | ❌ | ✅ Verify own code | ✅ Primary | ✅ Run quality gate | -| **Documentation** | ✅ PRD/tasks | ✅ Code comments | ✅ Test docs | ✅ Primary | -| **Quality Gate** | ❌ | ❌ | ❌ | ✅ Primary | -| **Cleanup** | ❌ | ❌ | ❌ | ✅ MANDATORY | -| **Multi-Model Consensus** | ✅ Primary | ✅ Complex decisions | ❌ | ❌ | -| **Workspace Management** | ✅ Create | ✅ Update | ✅ Update | ✅ Archive & Clean | - -## Workflow Phases - -### Phase 1: Planning (`/plan`) - -**Agent:** Planner -**Purpose:** Research-grounded planning and workspace creation - -**Steps:** +```python +# 1. Research first +Read("docs/guides/...") +mcp__context7__get-library-docs(...) +WebSearch(query="...") -1. Research guides, Context7, WebSearch -2. Create structured plan with zen.planner -3. Get consensus on complex decisions (zen.consensus) -4. Create workspace in `requirements/{requirement}/` -5. Write PRD, tasks, research, recovery docs +# 2. Use zen.planner for structured planning +mcp__zen__planner( + step="Plan vector search implementation", + step_number=1, + total_steps=6, + next_step_required=True +) -**Output:** +# 3. Get consensus on complex decisions +mcp__zen__consensus( + step="Evaluate: Protocol vs ABC for driver base", + models=[ + {"model": "gemini-2.5-pro", "stance": "neutral"}, + {"model": "openai/gpt-5", "stance": "neutral"} + ], + relevant_files=["sqlspec/protocols.py"], + next_step_required=False +) +# 4. Create workspace +Write("specs/active/{requirement}/prd.md", ...) +Write("specs/active/{requirement}/tasks.md", ...) +Write("specs/active/{requirement}/research/plan.md", ...) +Write("specs/active/{requirement}/recovery.md", ...) ``` -requirements/{requirement-slug}/ -├── prd.md # Product Requirements Document -├── tasks.md # Implementation checklist -├── research/ # Research findings -│ └── plan.md # Detailed plan -├── tmp/ # Temporary files -└── recovery.md # Session resume guide -``` - -**Hand off to:** Expert agent for implementation - -### Phase 2: Implementation (`/implement`) - -**Agent:** Expert -**Purpose:** Write clean, type-safe, performant code - -**Steps:** - -1. Read workspace (prd.md, tasks.md, research/plan.md) -2. Research implementation details (guides, Context7) -3. Implement following CLAUDE.md standards -4. Run tests to verify -5. Update workspace (tasks.md, recovery.md) - -**Tools Used:** - -- zen.debug (systematic debugging) -- zen.thinkdeep (complex decisions) -- zen.analyze (code analysis) -- Context7 (library docs) - -**Output:** - -- Production code in sqlspec/ -- Updated workspace files - -**Hand off to:** Testing agent for comprehensive tests - -### Phase 3: Testing (`/test`) - -**Agent:** Testing -**Purpose:** Create comprehensive unit and integration tests - -**Steps:** - -1. Read implementation -2. Consult testing guide -3. Create unit tests (tests/unit/) -4. Create integration tests (tests/integration/) -5. Test edge cases -6. Verify coverage (80%+ adapters, 90%+ core) -7. Update workspace - -**Output:** -- Unit tests in tests/unit/ -- Integration tests in tests/integration/ -- Updated workspace files +**Output:** Complete workspace in `specs/active/{requirement}/` -**Hand off to:** Docs & Vision for documentation and quality gate +### For Expert Agent -### Phase 4: Review (`/review`) +**Responsibilities:** -**Agent:** Docs & Vision -**Purpose:** Documentation, quality gate, and MANDATORY cleanup +- Implementation following AGENTS.md standards +- Auto-invoke Testing agent when implementation complete +- Auto-invoke Docs & Vision agent after tests pass +- Orchestrate entire development lifecycle -**3 Sequential Phases:** - -1. **Documentation:** - - Update docs/guides/ - - Update API reference - - Build docs locally - -2. **Quality Gate (MANDATORY):** - - Run `make lint` (must pass) - - Check for anti-patterns (hasattr, workaround naming, class tests) - - Run full test suite (must pass) - - Verify PRD acceptance criteria - - **BLOCKS if quality gate fails** - -3. **Cleanup (MANDATORY):** - - Remove all tmp/ directories - - Archive requirement to requirements/archive/ - - Keep only last 3 active requirements - - Archive planning reports - -**Output:** - -- Complete documentation -- Clean workspace -- Archived requirement -- Work ready for PR/commit - -## Agent Invocation Patterns - -### Planner Invoking Consensus - -For complex architectural decisions: +**Workflow:** ```python -Task( - subagent_type="general-purpose", - description="Get multi-model consensus on API design", - prompt=""" -Use zen.consensus to get multi-model agreement: +# 1. Read the plan +Read("specs/active/{requirement}/prd.md") +Read("specs/active/{requirement}/tasks.md") +Read("specs/active/{requirement}/research/plan.md") -Question: Should we use Protocol or ABC for driver base class? +# 2. Research implementation details +Read(f"docs/guides/adapters/{adapter}.md") +Read("docs/guides/performance/sqlglot-best-practices.md") +Read("AGENTS.md") # Code quality standards +mcp__context7__get-library-docs(...) # Library-specific docs -Models to consult: -- gemini-2.5-pro (neutral stance) -- openai/gpt-5 (neutral stance) +# 3. Implement with quality standards (see AGENTS.md) +Edit(file_path="...", old_string="...", new_string="...") -Include relevant files for context: -- sqlspec/protocols.py -- sqlspec/adapters/asyncpg/driver.py +# 4. Self-test +Bash(command="uv run pytest tests/integration/test_adapters/test_asyncpg/ -v") +Bash(command="make lint") -Write consensus findings to requirements/{requirement}/research/consensus.md +# 5. AUTO-INVOKE Testing Agent (MANDATORY) +Task( + subagent_type="testing", + description="Create comprehensive test suite", + prompt=f""" +Create comprehensive tests for specs/active/{requirement}. + +Requirements: +1. Read specs/active/{requirement}/prd.md for acceptance criteria +2. Create unit tests for all new functionality +3. Create integration tests for affected adapters +4. Test edge cases (empty, errors, boundaries) +5. Achieve >80% coverage +6. Update specs/active/{requirement}/tasks.md +7. Update specs/active/{requirement}/recovery.md + +All tests must pass before returning control. """ ) -``` - -### Expert Invoking Debugging - -For systematic debugging: -```python +# 6. AUTO-INVOKE Docs & Vision Agent (MANDATORY) Task( - subagent_type="general-purpose", - description="Debug asyncpg connection pool deadlock", - prompt=""" -Use zen.debug for systematic debugging: - -Problem: AsyncPG connection pool deadlocks under high load - -Use zen.debug to: -1. State hypothesis about root cause -2. Investigate code paths -3. Check for leaked connections -4. Verify pool configuration -5. Test fix - -Write findings to requirements/{requirement}/tmp/debug-{issue}.md + subagent_type="docs-vision", + description="Documentation, quality gate, knowledge capture, archive", + prompt=f""" +Complete 5-phase workflow for specs/active/{requirement}: + +Phase 1 - Documentation: +• Update Sphinx documentation +• Create/update guides in docs/guides/ +• Validate code examples +• Build docs without errors + +Phase 2 - Quality Gate: +• Verify all PRD acceptance criteria met +• Verify all tests passing +• Check AGENTS.md standards compliance +• BLOCK if any criteria not met + +Phase 3 - Knowledge Capture (MANDATORY): +• Analyze implementation for new patterns +• Extract best practices and conventions +• Update AGENTS.md with new patterns/examples +• Update relevant guides in docs/guides/ +• Document with working code examples + +Phase 4 - Re-validation (MANDATORY): +• Re-run tests after documentation updates +• Rebuild documentation +• Check pattern consistency +• Verify no breaking changes +• BLOCK if re-validation fails + +Phase 5 - Cleanup & Archive (MANDATORY): +• Remove all tmp/ directories +• Move specs/active/{requirement} to specs/archive/ +• Generate completion report + +Return comprehensive summary when complete. """ ) -``` - -### Testing Invoking Test Generation - -Testing agent is usually NOT invoked by other agents - it's invoked directly via `/test` command. - -### Docs & Vision Blocking on Quality Gate - -Quality gate BLOCKS completion if standards not met: -```markdown -❌ QUALITY GATE FAILED - -Issues found: -- 3 files with hasattr() defensive patterns -- 2 tests using class-based structure -- make lint has 5 errors - -⚠️ WORK NOT COMPLETE -Do NOT proceed to cleanup phase. -Fix issues above and re-run quality gate. +# 7. Update workspace +Edit("specs/active/{requirement}/tasks.md", ...) +Edit("specs/active/{requirement}/recovery.md", ...) ``` -## MCP Tools Matrix +**IMPORTANT:** Expert MUST NOT mark work complete until Docs & Vision agent confirms: -### Tool: zen.planner +- Quality gate passed +- Knowledge captured in AGENTS.md and guides +- Spec archived to specs/archive/ -**Who uses:** Planner agent -**Purpose:** Structured, multi-step planning -**When:** Creating detailed implementation plans +### For Testing Agent -**Example:** +**Responsibilities:** -```python -mcp__zen__planner( - step="Plan vector search implementation for Oracle and PostgreSQL", - step_number=1, - total_steps=6, - next_step_required=True -) -``` - -### Tool: zen.consensus +- Create comprehensive unit tests +- Create integration tests for all affected adapters +- Test edge cases and error conditions +- Achieve required coverage (>80% adapters, >90% core) -**Who uses:** Planner, Expert -**Purpose:** Multi-model decision verification -**When:** Complex architectural decisions, significant API changes +**Auto-invoked by:** Expert agent -**Example:** +**Workflow:** ```python -mcp__zen__consensus( - step="Evaluate: Protocol vs ABC for driver base class", - models=[ - {"model": "gemini-2.5-pro", "stance": "neutral"}, - {"model": "openai/gpt-5", "stance": "neutral"} - ], - relevant_files=["sqlspec/protocols.py"], - next_step_required=False -) -``` +# 1. Read implementation context +Read("specs/active/{requirement}/prd.md") +Read("specs/active/{requirement}/recovery.md") +Read("docs/guides/testing/testing.md") -### Tool: zen.debug +# 2. Create unit tests +Write("tests/unit/test_{module}/test_{feature}.py", ...) -**Who uses:** Expert -**Purpose:** Systematic debugging workflow -**When:** Complex bugs, mysterious errors, performance issues +# 3. Create integration tests +Write("tests/integration/test_adapters/test_{adapter}/test_{feature}.py", ...) -**Example:** +# 4. Verify all tests pass +Bash(command="uv run pytest -n 2 --dist=loadgroup") +Bash(command="uv run pytest --cov") -```python -mcp__zen__debug( - step="Investigate memory leak in long-running connections", - step_number=1, - total_steps=5, - hypothesis="Result cache holds strong references to connection objects", - findings="Cache never evicts old entries", - confidence="medium", - next_step_required=True -) +# 5. Update workspace +Edit("specs/active/{requirement}/tasks.md", ...) +Edit("specs/active/{requirement}/recovery.md", ...) ``` -### Tool: zen.thinkdeep +**Must verify:** All tests passing before returning control to Expert -**Who uses:** Expert -**Purpose:** Deep analysis for complex decisions -**When:** Architecture decisions, complex refactoring +### For Docs & Vision Agent -**Example:** +**Responsibilities:** -```python -mcp__zen__thinkdeep( - step="Analyze if we should use Protocol vs ABC for driver base class", - step_number=1, - total_steps=3, - hypothesis="Protocol better for runtime type checking without inheritance", - findings="Protocols work with type guards, avoid diamond problem", - focus_areas=["architecture", "performance"], - confidence="high", - next_step_required=True -) -``` +- Update documentation (Sphinx, guides) +- Quality gate validation (BLOCKS if fails) +- **Knowledge capture** - Update AGENTS.md and guides with new patterns +- **Re-validation** - Verify consistency after documentation updates +- Cleanup and archive (MANDATORY) -### Tool: zen.analyze +**Auto-invoked by:** Expert agent -**Who uses:** Expert -**Purpose:** Code analysis (architecture, performance, security) -**When:** Code review, performance optimization, security audit +**5-Phase Workflow:** -**Example:** +#### Phase 1: Documentation ```python -mcp__zen__analyze( - step="Analyze oracle adapter for performance bottlenecks", - step_number=1, - total_steps=3, - analysis_type="performance", - findings="Found N+1 query pattern in result mapping", - confidence="high", - next_step_required=True -) -``` +# Update API reference +Edit("docs/reference/adapters.rst", ...) -### Tool: Context7 +# Create/update guides +Write("docs/guides/{category}/{guide}.md", ...) -**Who uses:** All agents -**Purpose:** Get up-to-date library documentation -**When:** Need current API reference for libraries (asyncpg, oracledb, etc.) +# Build docs +Bash(command="make docs") +``` -**Example:** +#### Phase 2: Quality Gate (BLOCKS IF FAILS) -```python -# Step 1: Resolve library ID -mcp__context7__resolve-library-id(libraryName="asyncpg") +```bash +# Run all checks - MUST PASS +make lint +uv run pytest -n 2 --dist=loadgroup -# Step 2: Get docs -mcp__context7__get-library-docs( - context7CompatibleLibraryID="/MagicStack/asyncpg", - topic="connection pooling" -) +# Verify PRD acceptance criteria +# Check AGENTS.md standards compliance +# BLOCK if any failures ``` -### Tool: WebSearch - -**Who uses:** All agents -**Purpose:** Research current best practices (2025+) -**When:** Need recent best practices, database-specific patterns - -**Example:** +#### Phase 3: Knowledge Capture (MANDATORY - NEW) ```python -WebSearch(query="PostgreSQL 16 connection pooling best practices 2025") -``` +# 1. Analyze implementation for new patterns +Read("sqlspec/adapters/{adapter}/{module}.py") -## Workspace Management +# 2. Extract patterns and add to AGENTS.md +Edit( + file_path="AGENTS.md", + old_string="### Compliance Table", + new_string="""### New Pattern: {Pattern Name} -### Structure +When implementing {pattern}: +```python +# Example code showing the pattern +class ExampleClass: + def example_method(self): + # Pattern implementation + pass ``` -requirements/ -├── {requirement-1}/ # Active requirement -│ ├── prd.md -│ ├── tasks.md -│ ├── recovery.md -│ ├── research/ -│ │ └── plan.md -│ └── tmp/ # Cleaned by Docs & Vision -├── {requirement-2}/ # Active requirement -├── {requirement-3}/ # Active requirement -├── archive/ # Completed requirements -│ └── {old-requirement}/ -└── README.md -``` - -### Cleanup Protocol (MANDATORY) - -**When:** After every `/review` (Docs & Vision agent) - -**Steps:** - -1. Remove all tmp/ directories: - ```bash - find requirements/*/tmp -type d -exec rm -rf {} + - ``` +**Why this pattern:** -2. Archive completed requirement: +- Reason 1 +- Reason 2 - ```bash - mv requirements/{requirement} requirements/archive/{requirement} - ``` +**Example from {adapter}:** +See `sqlspec/adapters/{adapter}/{file}.py:{line}` -3. Keep only last 3 active requirements: +### Compliance Table""" - ```bash - # If more than 3 active, move oldest to archive - ``` - -**This is MANDATORY - never skip cleanup.** - -### Session Continuity - -To resume work across sessions/context resets: - -```python -# 1. List active requirements -Glob("requirements/*/prd.md") +) -# 2. Read recovery.md to understand status -Read("requirements/{requirement}/recovery.md") +# 3. Update relevant guides -# 3. Check task progress -Read("requirements/{requirement}/tasks.md") +Edit("docs/guides/{category}/{guide}.md", ...) -# 4. Review PRD for full context -Read("requirements/{requirement}/prd.md") +# 4. Document with working examples -# 5. Review planning details -Read("requirements/{requirement}/research/plan.md") ``` -## Code Quality Standards - -All agents MUST enforce CLAUDE.md standards: - -### ✅ ALWAYS DO +**What to capture in AGENTS.md:** +- New design patterns discovered +- Performance optimizations applied +- Type annotation patterns +- Error handling approaches +- Testing strategies +- Database-specific patterns -- **Type hints:** Stringified for custom types: `def foo(config: "SQLConfig"):` -- **Type guards:** `if supports_where(obj):` from `sqlspec.utils.type_guards` -- **Clean names:** `process_query()`, `execute_batch()` -- **Top-level imports:** Except TYPE_CHECKING -- **Function-based tests:** `def test_something():` -- **Early returns:** Guard clauses for edge cases -- **Functions under 75 lines:** Extract helpers if longer +**How to update AGENTS.md:** +- Add to existing sections (don't create new top-level sections) +- Include working code examples +- Reference actual files with line numbers +- Explain WHY not just WHAT -### ❌ NEVER DO +#### Phase 4: Re-validation (MANDATORY - NEW) -- **Future annotations:** `from __future__ import annotations` -- **Defensive patterns:** `hasattr()`, `getattr()` -- **Workaround naming:** `_optimized`, `_with_cache`, `_fallback` -- **Nested imports:** Except TYPE_CHECKING -- **Class-based tests:** `class TestSomething:` -- **Magic numbers:** Use named constants -- **Comments:** Use docstrings instead - -## Guides Reference +```bash +# Re-run tests after doc updates +uv run pytest -n 2 --dist=loadgroup -All agents should consult guides before implementing: +# Rebuild docs +make docs -### Adapter Guides +# Verify pattern consistency across project +# Check no breaking changes introduced -``` -docs/guides/adapters/ -├── adbc.md -├── aiosqlite.md -├── asyncmy.md -├── bigquery.md -├── duckdb.md -├── mysql.md -├── oracle.md # Most comprehensive -├── postgres.md -├── psqlpy.md -└── sqlite.md +# BLOCK if re-validation fails ``` -### Performance Guides +#### Phase 5: Cleanup & Archive (MANDATORY) -``` -docs/guides/performance/ -├── sqlglot-best-practices.md -├── sqlglot-cheat-sheet.md -├── mypyc-optimizations.md -└── mypyc-guide.md -``` +```bash +# Remove tmp files +find specs/active/{requirement}/tmp -type f -delete +rmdir specs/active/{requirement}/tmp -### Architecture Guides +# Archive requirement +mv specs/active/{requirement} specs/archive/{requirement} -``` -docs/guides/architecture/ -├── architecture.md -└── data-flow.md +# Generate completion report ``` -### Testing Guide +## Workspace Structure ``` -docs/guides/testing/ -└── testing.md +specs/ +├── active/ # Current work (gitignored) +│ └── {requirement}/ +│ ├── prd.md # Product requirements +│ ├── tasks.md # Implementation checklist +│ ├── recovery.md # Session resume guide +│ ├── research/ # Research findings +│ │ └── plan.md +│ └── tmp/ # Temp files (cleaned by Docs & Vision) +├── archive/ # Completed work (gitignored) +│ └── {old-requirement}/ +└── template-spec/ # Template files (committed) + ├── prd.md + ├── tasks.md + ├── recovery.md + └── README.md ``` -### Quick Reference +## Cross-Agent Patterns -``` -docs/guides/quick-reference/ -└── quick-reference.md -``` - -## Recovery Patterns +### Session Continuity -### After Context Reset +To resume work after context reset: ```python # 1. Find active work -active_requirements = Glob("requirements/*/prd.md") +Glob("specs/active/*/prd.md") -# 2. For each active requirement, check status -for req_prd in active_requirements: - req_dir = req_prd.parent - recovery = Read(f"{req_dir}/recovery.md") - # Shows: Status, Last updated, Next steps +# 2. Read recovery.md for each +Read("specs/active/{requirement}/recovery.md") # 3. Resume from most recent -Read("{most_recent_requirement}/recovery.md") # Clear next steps -Read("{most_recent_requirement}/tasks.md") # See what's done +Read("specs/active/{requirement}/tasks.md") ``` -### After Session Timeout +### Quality Standards -Same as context reset - recovery.md has all needed info. +All agents enforce AGENTS.md standards: -### After Cleanup +✅ **ALWAYS:** -If requirement archived: +- Stringified type hints: `def foo(config: "SQLConfig"):` +- Type guards: `if supports_where(obj):` +- Function-based tests: `def test_something():` +- Functions under 75 lines -```python -# Find in archive -Glob("requirements/archive/*/prd.md") +❌ **NEVER:** -# Can still read archived requirements -Read("requirements/archive/{requirement}/recovery.md") -``` - -## Command Workflow +- `from __future__ import annotations` +- Defensive patterns: `hasattr()`, `getattr()` +- Class-based tests: `class TestSomething:` +- Nested imports (except TYPE_CHECKING) -### Full Feature Development +See `AGENTS.md` for complete standards. -```bash -# 1. Plan -/plan implement vector search for Oracle and PostgreSQL +## Model-Specific Instructions -# Creates: requirements/vector-search/ +### For Codex -# 2. Implement -/implement +Codex can emulate slash commands by following agent workflows: -# Modifies code, updates workspace +**To run implementation phase:** -# 3. Test -/test +``` +Follow every step in expert.md workflow, then auto-invoke Testing and Docs & Vision agents as subagents. +Always read specs/active/{requirement}/ before making changes. +``` -# Creates tests, verifies passing +See `.claude/agents/expert.md` lines 22-23 for details. -# 4. Review (3 phases: docs → quality gate → cleanup) -/review +### For Gemini -# Phase 1: Documentation -# Phase 2: Quality gate (must pass) -# Phase 3: Cleanup (mandatory) +Gemini can use the same workflow patterns. When asked to "implement feature X": -# Result: requirements/vector-search/ → requirements/archive/vector-search/ -``` +1. **Check for existing workspace**: `Glob("specs/active/*/prd.md")` +2. **If workspace exists**: Follow Expert workflow (read plan → research → implement → auto-invoke Testing → auto-invoke Docs & Vision) +3. **If no workspace**: Suggest user run `/plan` first, or create minimal workspace yourself -### Bug Fix Workflow +**Gemini-specific macro for implementation:** -```bash -# 1. Plan (optional for simple bugs) -/plan fix connection pool deadlock in asyncpg +``` +You are implementing {feature} for SQLSpec. Follow this workflow: -# 2. Debug and implement -/implement +PHASE 1 - Read Plan: +• Read specs/active/{requirement}/prd.md +• Read specs/active/{requirement}/tasks.md +• Read specs/active/{requirement}/research/plan.md -# Expert uses zen.debug for systematic investigation +PHASE 2 - Research: +• Read AGENTS.md for code standards +• Read docs/guides/adapters/{adapter}.md +• Read docs/guides/performance/sqlglot-best-practices.md +• Use zen.chat or Context7 for library docs -# 3. Test -/test +PHASE 3 - Implement: +• Follow AGENTS.md standards ruthlessly +• Self-test with: uv run pytest tests/... +• Update specs/active/{requirement}/tasks.md -# Add regression test +PHASE 4 - Auto-Invoke Testing Agent: +• Use zen.chat with testing agent instructions +• Pass requirement name and acceptance criteria +• Wait for confirmation all tests pass -# 4. Review -/review +PHASE 5 - Auto-Invoke Docs & Vision Agent: +• Use zen.chat with docs-vision agent instructions +• Pass requirement name +• Wait for 5-phase completion (docs, QA, knowledge, re-validation, archive) -# Quality gate + cleanup +PHASE 6 - Complete: +• Verify spec archived to specs/archive/ +• Return comprehensive summary ``` -## Best Practices +### For Claude Code -### For Planner +Claude Code uses native slash commands: -1. **Always research first** - guides, Context7, WebSearch -2. **Use zen.planner** for complex work -3. **Get consensus** on significant decisions -4. **Create complete workspace** - don't skip files -5. **Write clear recovery.md** - enable easy resume +- `/plan {feature}` - Invokes Planner agent +- `/implement` - Invokes Expert agent (auto-invokes Testing and Docs & Vision) +- `/test` - Directly invokes Testing agent (rarely needed) +- `/review` - Directly invokes Docs & Vision agent (rarely needed) -### For Expert +## MCP Tools Reference -1. **Read the plan first** - don't guess -2. **Consult guides** - adapters, performance, architecture -3. **Use zen tools** for complex work (debug, thinkdeep, analyze) -4. **Follow CLAUDE.md** ruthlessly -5. **Update workspace** continuously -6. **Test as you go** - don't wait for Testing agent +### zen.planner -### For Testing +- **Who**: Planner agent +- **When**: Structured multi-step planning +- **Pattern**: Iterative steps with branching/revision -1. **Consult testing guide** before creating tests -2. **Function-based tests** always (no classes) -3. **Mark appropriately** - @pytest.mark.asyncio, @pytest.mark.postgres, etc. -4. **Test edge cases** - empty, None, errors, concurrency -5. **Verify coverage** - 80%+ adapters, 90%+ core -6. **All tests must pass** before handoff +### zen.consensus -### For Docs & Vision +- **Who**: Planner, Expert +- **When**: Complex architectural decisions +- **Pattern**: Multi-model consultation with stances -1. **Phase 1 (Docs)** - Comprehensive and clear -2. **Phase 2 (Quality Gate)** - BLOCK if standards not met -3. **Phase 3 (Cleanup)** - MANDATORY, never skip -4. **Archive systematically** - maintain clean workspace -5. **Final verification** - one last `make lint && make test` +### zen.debug -## Troubleshooting +- **Who**: Expert +- **When**: Systematic debugging, root cause analysis +- **Pattern**: Hypothesis-driven investigation -### Quality Gate Failing +### zen.thinkdeep -```markdown -**Problem:** Quality gate keeps failing +- **Who**: Expert +- **When**: Deep analysis for complex decisions +- **Pattern**: Multi-step reasoning with evidence -**Solution:** -1. Check specific failure reasons -2. Fix anti-patterns (hasattr, workaround naming, class tests) -3. Run `make fix` to auto-fix lint issues -4. Re-run quality gate -5. DO NOT proceed to cleanup until passing -``` +### zen.analyze -### Workspace Getting Cluttered +- **Who**: Expert +- **When**: Code analysis (architecture, performance, security) +- **Pattern**: Systematic code review -```markdown -**Problem:** requirements/ has too many folders +### zen.chat -**Solution:** -1. Run `/review` on oldest requirements -2. Let Docs & Vision archive them -3. Manually archive if needed: - mv requirements/{old-requirement} requirements/archive/ -4. Keep only 3 active requirements -``` +- **Who**: Any agent (especially Gemini) +- **When**: General collaboration, brainstorming, sub-agent emulation +- **Pattern**: Conversational with context files -### Lost Context Across Sessions +### Context7 -```markdown -**Problem:** Can't remember what I was working on +- **Who**: All agents +- **When**: Need library documentation +- **Pattern**: Resolve library ID → Get docs with topic -**Solution:** -1. Read requirements/*/recovery.md for all active requirements -2. Each recovery.md has: - - Current status - - Last updated date - - Next steps -3. Resume from most recent -``` +### WebSearch + +- **Who**: All agents +- **When**: Research current best practices (2025+) +- **Pattern**: Search query with date filtering ## Summary -This agent system provides: +This workflow ensures: -✅ **Structured workflow** - Plan → Implement → Test → Review -✅ **Quality enforcement** - CLAUDE.md standards mandatory -✅ **Research-grounded** - Guides + Context7 + WebSearch -✅ **Session continuity** - Workspace enables resume -✅ **Cleanup protocol** - Mandatory workspace management -✅ **MCP tool integration** - zen.planner, zen.debug, zen.consensus, Context7 +✅ **Automated orchestration** - Expert agent handles entire lifecycle +✅ **Knowledge preservation** - Every feature updates AGENTS.md and guides +✅ **Quality assurance** - Multi-phase validation before completion +✅ **Session continuity** - Workspace enables cross-session resume +✅ **Multi-model support** - Works with Claude Code, Gemini, Codex -All agents work together to ensure high-quality, well-tested, well-documented code that follows SQLSpec's strict standards. +The key innovation: **Knowledge Capture (Phase 3)** and **Re-validation (Phase 4)** ensure every feature improves the project's documentation and maintains consistency. diff --git a/.claude/agents/docs-vision.md b/.claude/agents/docs-vision.md index 992092145..6d7008cb2 100644 --- a/.claude/agents/docs-vision.md +++ b/.claude/agents/docs-vision.md @@ -7,23 +7,27 @@ model: sonnet # Docs & Vision Agent -Triple-responsibility agent combining documentation excellence, quality gate validation, and mandatory workspace cleanup. +Five-phase agent combining documentation excellence, quality gate validation, knowledge capture, re-validation, and mandatory workspace cleanup. ## Core Responsibilities 1. **Documentation** - Write/update comprehensive documentation 2. **Quality Gate** - Validate code quality before completion -3. **Cleanup** - Clean workspace, archive completed work +3. **Knowledge Capture** - Extract patterns and update AGENTS.md and guides +4. **Re-validation** - Verify consistency after knowledge updates +5. **Cleanup & Archive** - Clean workspace, archive completed work ## Workflow Overview -This agent runs in **3 sequential phases**: +Codex or Gemini CLI may run this workflow end-to-end without invoking `/review`. When asked to “complete docs, quality gate, and cleanup” for a workspace, either assistant must execute all five phases exactly as detailed below, update AGENTS.md and guides during knowledge capture, and finish with archival. Claude should continue to invoke `/review` unless manually directed otherwise. + +This agent runs in **5 sequential phases**: ``` -Phase 1: Documentation → Phase 2: Quality Gate → Phase 3: Cleanup +Phase 1: Documentation → Phase 2: Quality Gate → Phase 3: Knowledge Capture → Phase 4: Re-validation → Phase 5: Cleanup & Archive ``` -All 3 phases MUST complete before work is considered done. +All 5 phases MUST complete before work is considered done. --- @@ -37,8 +41,8 @@ Understand what needs documenting: ```python # Read workspace -Read("requirements/{requirement}/prd.md") -Read("requirements/{requirement}/tasks.md") +Read("specs/active/{requirement}/prd.md") +Read("specs/active/{requirement}/tasks.md") # Read implementation Read("sqlspec/adapters/asyncpg/driver.py") @@ -286,7 +290,7 @@ uv run pytest -n 2 --dist=loadgroup **Check acceptance criteria:** ```python -Read("requirements/{requirement}/prd.md") +Read("specs/active/{requirement}/prd.md") # Manually verify each criterion: # - [ ] Feature works as described @@ -335,14 +339,258 @@ return ```python print("✅ QUALITY GATE PASSED") -print("\n Proceeding to Phase 3: Cleanup") +print("\nProceeding to Phase 3: Knowledge Capture") +``` + +--- + +## Phase 3: Knowledge Capture (NEW!) + +**Extract new patterns from implementation and update AGENTS.md and guides.** + +This phase captures organizational learning so future implementations benefit from discoveries. + +### Step 1: Analyze Implementation for Patterns + +```python +# Read implementation details +Read("specs/active/{requirement}/recovery.md") +Read("specs/active/{requirement}/research/") + +# Review what was implemented +Grep(pattern="class.*Config|class.*Driver|def.*handler", path="sqlspec/adapters/", output_mode="content", head_limit=50) +``` + +**Look for:** + +1. **New Patterns**: Novel approaches to common problems +2. **Best Practices**: Techniques that worked particularly well +3. **Conventions**: Naming, structure, or organization patterns +4. **Type Handling**: New type conversion or validation approaches +5. **Testing Patterns**: Effective test strategies +6. **Performance Techniques**: Optimization discoveries +7. **Error Handling**: Robust error management patterns + +### Step 2: Update AGENTS.md with New Patterns + +**Add patterns to relevant sections:** + +```python +# Read current AGENTS.md +current_content = Read("AGENTS.md") + +# Example: Add new driver_features pattern +Edit( + file_path="AGENTS.md", + old_string="### Compliance Table\n\nCurrent state of all adapters", + new_string="""### New Pattern: Session Callbacks for Type Handlers + +When implementing optional type handlers (NumPy, pgvector, etc.): + +```python +class AdapterConfig(AsyncDatabaseConfig): + async def _create_pool(self): + config = dict(self.pool_config) + + if self.driver_features.get("enable_feature", False): + config["session_callback"] = self._init_connection + + return await create_pool(**config) + + async def _init_connection(self, connection): + if self.driver_features.get("enable_feature", False): + from ._feature_handlers import register_handlers + register_handlers(connection) +``` + +This pattern: + +- Lazily imports type handlers only when needed +- Registers handlers per-connection for safety +- Allows graceful degradation when dependencies missing + +### Compliance Table + +Current state of all adapters""" +) + +``` + +**Common sections to update:** + +- **Code Quality Standards** - New coding patterns +- **Testing Strategy** - New test approaches +- **Performance Optimizations** - New optimization techniques +- **Database Adapter Implementation** - Adapter-specific patterns +- **driver_features Pattern** - New feature configurations + +### Step 3: Update Guides with New Patterns + +**Enhance relevant guides in docs/guides/:** + +```python +# Example: Update adapter guide +Edit( + file_path="docs/guides/adapters/postgres.md", + old_string="## Advanced Features", + new_string="""## Advanced Features + +### Automatic pgvector Support + +PostgreSQL adapters now auto-detect and enable pgvector when installed: + +```python +from sqlspec.adapters.asyncpg.config import AsyncpgConfig + +# pgvector automatically enabled if installed +config = AsyncpgConfig(dsn="postgresql://localhost/db") + +async with config.provide_session() as session: + # Vectors work automatically + embedding = [0.1, 0.2, 0.3, ...] + await session.execute( + "INSERT INTO embeddings (id, vector) VALUES ($1, $2)", + (1, embedding) + ) +``` + +This leverages the `driver_features` auto-detection pattern for seamless integration. +""" +) + +``` + +**Guides to consider:** + +- `docs/guides/adapters/{adapter}.md` - Adapter-specific patterns +- `docs/guides/testing/testing.md` - Testing patterns +- `docs/guides/performance/` - Performance techniques +- `docs/guides/architecture/` - Architectural patterns + +### Step 4: Document with Working Examples + +**Ensure all new patterns have working code examples:** + +```python +# Test the example code +Bash(command=""" +cat > /tmp/test_pattern.py << 'EOF' +from sqlspec.adapters.asyncpg.config import AsyncpgConfig + +config = AsyncpgConfig(dsn="postgresql://localhost/test") +print(config.driver_features) +EOF + +uv run python /tmp/test_pattern.py +""") ``` +**If example doesn't work, fix it before adding to docs.** + --- -## Phase 3: Cleanup (MANDATORY) +## Phase 4: Re-validation (NEW!) + +**Verify consistency and stability after knowledge capture updates.** + +This phase ensures documentation updates didn't break anything. + +### Step 1: Re-run Tests + +```bash +# Run full test suite again +uv run pytest -n 2 --dist=loadgroup + +# Should see: +# ===== X passed in Y.YYs ===== +``` + +**If tests fail after doc updates:** + +1. Identify what broke +2. Fix the issue (likely in AGENTS.md or guides) +3. Re-run tests +4. **DO NOT PROCEED** until tests pass + +### Step 2: Rebuild Documentation + +```bash +# Rebuild docs to catch any errors introduced +make docs + +# Should see: +# build succeeded, X warnings. +``` + +**Fix any new warnings or errors:** + +- Broken cross-references from new content +- Invalid RST syntax in updates +- Missing files referenced in new examples + +### Step 3: Verify Pattern Consistency + +```python +# Check that new patterns don't contradict existing ones +Read("AGENTS.md") + +# Manually verify: +# - New patterns align with existing standards +# - No contradictory advice +# - Examples follow project conventions +# - Terminology is consistent +``` + +### Step 4: Check for Breaking Changes + +```python +# Verify no breaking changes introduced +Grep(pattern="BREAKING|deprecated|removed", path="AGENTS.md", output_mode="content") +Grep(pattern="BREAKING|deprecated|removed", path="docs/guides/", output_mode="content") + +# If found, ensure properly documented and justified +``` + +### Step 5: Re-validation Decision -**This phase is MANDATORY after every quality gate pass.** +**Re-validation PASSES if:** +✅ All tests still passing +✅ Documentation builds without errors +✅ New patterns consistent with existing +✅ No unintended breaking changes +✅ Examples work as documented + +**Re-validation FAILS if:** +❌ Tests broken after updates +❌ Documentation build errors +❌ Pattern contradictions +❌ Undocumented breaking changes + +**If re-validation FAILS:** + +```python +print("❌ RE-VALIDATION FAILED") +print("\nIssues found:") +print("- 2 tests broken after AGENTS.md update") +print("- Documentation has 3 new warnings") +print("\n⚠️ FIX ISSUES BEFORE PROCEEDING") + +# STOP HERE - DO NOT PROCEED TO CLEANUP +return +``` + +**If re-validation PASSES:** + +```python +print("✅ RE-VALIDATION PASSED") +print("\nProceeding to Phase 5: Cleanup & Archive") +``` + +--- + +## Phase 5: Cleanup & Archive (MANDATORY) + +**This phase is MANDATORY after re-validation passes.** Cleanup workspace, archive completed work, remove temporary files. @@ -352,10 +600,10 @@ Cleanup workspace, archive completed work, remove temporary files. ```bash # Find and remove tmp directories -find requirements/*/tmp -type d -exec rm -rf {} + 2>/dev/null || true +find specs/active/*/tmp -type d -exec rm -rf {} + 2>/dev/null || true # Verify removed -find requirements/*/tmp 2>/dev/null +find specs/active/*/tmp 2>/dev/null # Should return nothing ``` @@ -363,13 +611,13 @@ find requirements/*/tmp 2>/dev/null ```bash # Remove verification artifacts -rm -rf requirements/verification/ 2>/dev/null || true +rm -rf specs/active/verification/ 2>/dev/null || true -# Remove any __pycache__ in requirements/ -find requirements -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true +# Remove any __pycache__ in specs/ +find specs -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true # Remove any .DS_Store or other cruft -find requirements -name ".DS_Store" -delete 2>/dev/null || true +find specs -name ".DS_Store" -delete 2>/dev/null || true ``` ### Step 2: Update Final Status @@ -378,9 +626,9 @@ find requirements -name ".DS_Store" -delete 2>/dev/null || true ```python Edit( - file_path="requirements/{requirement}/tasks.md", - old_string="- [ ] 5. Documentation", - new_string="- [x] 5. Documentation" + file_path="specs/active/{requirement}/tasks.md", + old_string="- [ ] 7. Archived (via Docs & Vision agent)", + new_string="- [x] 7. Archived (via Docs & Vision agent)" ) ``` @@ -388,12 +636,14 @@ Edit( ```python Edit( - file_path="requirements/{requirement}/recovery.md", - old_string="Status: Documentation", + file_path="specs/active/{requirement}/recovery.md", + old_string="Status: {current_status}", new_string="""Status: ✅ COMPLETE -Completion date: 2025-10-09 +Completion date: 2025-10-19 Quality gate: PASSED +Knowledge capture: COMPLETE +Re-validation: PASSED Tests: All passing Documentation: Complete """ @@ -406,60 +656,31 @@ Documentation: Complete ```bash # Create archive directory if needed -mkdir -p requirements/archive +mkdir -p specs/archive # Move completed requirement to archive -mv requirements/{requirement-slug} requirements/archive/{requirement-slug} +mv specs/active/{requirement-slug} specs/archive/{requirement-slug} # Verify archived -ls -la requirements/archive/{requirement-slug} -``` - -### Step 4: Clean requirements/ Root - -**Keep only last 3 active requirements:** - -```python -# List all non-archived requirements -active_reqs = Glob("requirements/*/prd.md") - -# If more than 3 active requirements -if len(active_reqs) > 3: - # Sort by modification time (oldest first) - # Move oldest to archive - for old_req in active_reqs[:-3]: - req_dir = old_req.parent - Bash(f"mv {req_dir} requirements/archive/") -``` - -### Step 5: Cleanup Reports - -**Archive planning reports:** - -```bash -# Move completed planning reports to archive -mkdir -p .claude/reports/archive/$(date +%Y-%m) - -mv .claude/reports/{requirement-name}-*.md .claude/reports/archive/$(date +%Y-%m)/ 2>/dev/null || true +ls -la specs/archive/{requirement-slug} ``` -### Step 6: Final Verification +### Step 4: Final Verification **Verify workspace is clean:** ```bash -# Check requirements/ structure -ls -la requirements/ +# Check specs/ structure +ls -la specs/ # Should show: -# - archive/ (archived requirements) -# - {active-req-1}/ (if any) -# - {active-req-2}/ (if any) -# - {active-req-3}/ (if any) +# - active/ (active specs) +# - archive/ (archived specs) +# - template-spec/ (template) # - README.md # No tmp/ directories should exist -find requirements -name tmp -type d +find specs -name tmp -type d # Should return nothing ``` @@ -467,7 +688,7 @@ find requirements -name tmp -type d ## Completion Report -After all 3 phases complete, provide summary: +After all 5 phases complete, provide summary: ```markdown # Work Complete: {Feature Name} @@ -485,21 +706,38 @@ After all 3 phases complete, provide summary: - Coverage: 87% (target: 80%) - PRD criteria: ✅ All met -## ✅ Cleanup (Phase 3) +## ✅ Knowledge Capture (Phase 3) +- Patterns extracted: 3 new patterns +- AGENTS.md updated: Added session callback pattern +- Guides updated: docs/guides/adapters/postgres.md +- Examples validated: ✅ All working + +## ✅ Re-validation (Phase 4) +- Tests after updates: ✅ 45/45 passing +- Documentation rebuild: ✅ No new errors +- Pattern consistency: ✅ Verified +- Breaking changes: ✅ None + +## ✅ Cleanup & Archive (Phase 5) - Temporary files: ✅ Removed -- Workspace: ✅ Archived to requirements/archive/{requirement} -- Reports: ✅ Archived to .claude/reports/archive/ -- requirements/ root: ✅ Clean (2 active requirements) +- Workspace: ✅ Archived to specs/archive/{requirement} +- specs/ root: ✅ Clean ## Files Modified - [sqlspec/adapters/asyncpg/driver.py](sqlspec/adapters/asyncpg/driver.py#L42-L67) - [sqlspec/core/result.py](sqlspec/core/result.py#L123) - [docs/guides/adapters/asyncpg.md](docs/guides/adapters/asyncpg.md) +- [AGENTS.md](AGENTS.md) - Knowledge capture ## Tests Added - [tests/integration/test_adapters/test_asyncpg/test_connection.py](tests/integration/test_adapters/test_asyncpg/test_connection.py) - [tests/unit/test_core/test_statement.py](tests/unit/test_core/test_statement.py) +## Knowledge Captured +- Session callback pattern for type handlers +- Auto-detection pattern for optional dependencies +- Per-connection handler registration approach + ## Next Steps Feature complete and ready for PR! 🎉 @@ -572,7 +810,10 @@ def test_asyncpg_connection_basic(): ✅ **Phase 1 Complete** - Documentation comprehensive and builds ✅ **Phase 2 Complete** - Quality gate passed -✅ **Phase 3 Complete** - Workspace cleaned and archived +✅ **Phase 3 Complete** - Knowledge captured in AGENTS.md and guides +✅ **Phase 4 Complete** - Re-validation passed after updates +✅ **Phase 5 Complete** - Workspace cleaned and archived ✅ **All tests pass** - `make lint && make test` success ✅ **Standards followed** - CLAUDE.md compliance +✅ **Knowledge preserved** - Future implementations benefit ✅ **Clean handoff** - Ready for PR/commit diff --git a/.claude/agents/expert.md b/.claude/agents/expert.md index c2bab2d5c..4cd1d26a3 100644 --- a/.claude/agents/expert.md +++ b/.claude/agents/expert.md @@ -19,19 +19,21 @@ Domain expert for SQLSpec implementation. Handles all technical work: core devel ## Implementation Workflow +Codex or Gemini CLI can emulate this workflow without the `/implement` command. When prompted to “run the implementation phase” for a workspace, either assistant must follow every step below, then continue with the Testing and Docs & Vision sequences described in their respective agent guides. Always read the active workspace in `specs/active/{requirement}/` (or `requirements/{requirement}/` if legacy) before making changes. Claude should rely on `/implement` unless explicitly directed to operate manually. + ### Step 1: Read the Plan Always start by understanding the full scope: ```python # Read PRD from workspace -Read("requirements/{requirement}/prd.md") +Read("specs/active/{requirement}/prd.md") # Check tasks list -Read("requirements/{requirement}/tasks.md") +Read("specs/active/{requirement}/tasks.md") # Review research findings -Read("requirements/{requirement}/research/plan.md") +Read("specs/active/{requirement}/research/plan.md") ``` ### Step 2: Research Implementation Details @@ -207,26 +209,105 @@ make lint make fix ``` -### Step 6: Update Workspace +### Step 6: Auto-Invoke Testing Agent (MANDATORY) + +After implementation is complete, automatically invoke the Testing agent: + +```python +# Invoke Testing agent as subagent +Task( + subagent_type="testing", + description="Create comprehensive test suite", + prompt=f""" +Create comprehensive tests for the implemented feature in specs/active/{requirement}. + +Requirements: +1. Read specs/active/{requirement}/prd.md for acceptance criteria +2. Read specs/active/{requirement}/recovery.md for implementation details +3. Create unit tests for all new functionality +4. Create integration tests for all affected adapters +5. Test edge cases (empty results, errors, boundaries) +6. Achieve >80% coverage +7. Update specs/active/{requirement}/tasks.md marking test phase complete +8. Update specs/active/{requirement}/recovery.md with test results + +All tests must pass before returning control to Expert agent. +""" +) +``` + +### Step 7: Auto-Invoke Docs & Vision Agent (MANDATORY) + +After tests pass, automatically invoke the Docs & Vision agent: + +```python +# Invoke Docs & Vision agent as subagent +Task( + subagent_type="docs-vision", + description="Documentation, quality gate, knowledge capture, and archive", + prompt=f""" +Complete the documentation, quality gate, knowledge capture, and archival process for specs/active/{requirement}. + +Phase 1 - Documentation: +1. Read specs/active/{requirement}/prd.md for feature details +2. Update project documentation (Sphinx) +3. Create/update guides in docs/guides/ +4. Validate code examples work +5. Build documentation without errors + +Phase 2 - Quality Gate: +1. Verify all PRD acceptance criteria met +2. Verify all tests passing +3. Check code standards compliance (AGENTS.md) +4. BLOCK if any criteria not met + +Phase 3 - Knowledge Capture: +1. Analyze implementation for new patterns +2. Extract best practices and conventions +3. Update AGENTS.md with new patterns +4. Update relevant guides in docs/guides/ +5. Document patterns with working examples + +Phase 4 - Re-validation: +1. Re-run tests after documentation updates +2. Rebuild documentation to verify no errors +3. Check pattern consistency across project +4. Verify no breaking changes introduced +5. BLOCK if re-validation fails + +Phase 5 - Cleanup & Archive: +1. Remove all tmp/ files +2. Move specs/active/{requirement} to specs/archive/ +3. Generate completion report + +Return comprehensive completion summary when done. +""" +) +``` + +### Step 8: Update Workspace -Track progress in `requirements/{requirement}/`: +Track progress in `specs/active/{requirement}/`: ```markdown # In tasks.md, mark completed items: - [x] 2. Core implementation - [x] 3. Adapter-specific code -- [ ] 4. Testing ← UPDATE THIS +- [x] 4. Testing (via Testing agent) +- [x] 5. Documentation (via Docs & Vision agent) +- [x] 6. Knowledge Capture (via Docs & Vision agent) +- [x] 7. Archived (via Docs & Vision agent) ``` ```markdown # In recovery.md, update status: ## Current Status -Status: Testing -Last updated: 2025-10-09 +Status: Complete - archived +Last updated: 2025-10-19 -## Next Steps -- Complete integration tests for asyncpg -- Add test for edge case: empty result set +## Final Summary +Implementation, testing, documentation, and knowledge capture complete. +Spec archived to specs/archive/{requirement}/ ``` ## Database Adapter Implementation @@ -339,45 +420,42 @@ mcp__zen__debug( # Continue until root cause found... ``` -## Handoff to Testing Agent - -When implementation complete: - -1. **Mark tasks complete:** - - ```markdown - - [x] 2. Core implementation - - [x] 3. Adapter-specific code - - [ ] 4. Testing ← HAND OFF TO TESTING AGENT - ``` - -2. **Update recovery.md:** +## Automated Workflow - ```markdown - ## Current Status - Status: Ready for testing - Files modified: - - sqlspec/adapters/asyncpg/driver.py - - sqlspec/core/result.py +The Expert agent orchestrates a complete workflow: - ## Next Steps - Testing agent should: - - Add unit tests for new methods - - Add integration tests for asyncpg - - Verify edge cases handled - ``` - -3. **Notify user:** - - ``` - Implementation complete! +``` +┌─────────────────────────────────────────────────────────────┐ +│ EXPERT AGENT │ +│ │ +│ 1. Read Plan & Research │ +│ 2. Implement Feature │ +│ 3. Self-Test & Verify │ +│ 4. ──► Auto-Invoke Testing Agent (subagent) │ +│ │ │ +│ ├─► Create unit tests │ +│ ├─► Create integration tests │ +│ ├─► Test edge cases │ +│ └─► Verify coverage & all tests pass │ +│ 5. ──► Auto-Invoke Docs & Vision Agent (subagent) │ +│ │ │ +│ ├─► Update documentation │ +│ ├─► Quality gate validation │ +│ ├─► Update AGENTS.md with new patterns │ +│ ├─► Update guides with new patterns │ +│ ├─► Re-validate (tests, docs, consistency) │ +│ ├─► Clean tmp/ and archive │ +│ └─► Generate completion report │ +│ 6. Return Complete Summary │ +└─────────────────────────────────────────────────────────────┘ +``` - Modified files: - - [sqlspec/adapters/asyncpg/driver.py](sqlspec/adapters/asyncpg/driver.py#L42-L67) - - [sqlspec/core/result.py](sqlspec/core/result.py#L123) +**IMPORTANT**: The Expert agent MUST NOT mark implementation complete until: - Next: Invoke Testing agent to create comprehensive tests. - ``` +1. Testing agent confirms all tests pass +2. Docs & Vision agent confirms quality gate passed +3. Docs & Vision agent confirms knowledge captured in AGENTS.md and guides +4. Spec is properly archived to specs/archive/ ## Tools Available @@ -397,7 +475,7 @@ When implementation complete: # User: "Implement connection pooling for asyncpg" # 1. Read plan -Read("requirements/asyncpg-pooling/prd.md") +Read("specs/active/asyncpg-pooling/prd.md") # 2. Research Read("docs/guides/adapters/postgres.md") @@ -413,11 +491,14 @@ Edit( new_string="pool = await asyncpg.create_pool(**pool_config)" ) -# 4. Test +# 4. Test locally Bash(command="uv run pytest tests/integration/test_adapters/test_asyncpg/ -v") -# 5. Update workspace -Edit(file_path="requirements/asyncpg-pooling/tasks.md", ...) +# 5. Auto-invoke Testing agent (creates comprehensive tests) +Task(subagent_type="testing", description="Create test suite", prompt=...) + +# 6. Auto-invoke Docs & Vision agent (docs, QA, knowledge, archive) +Task(subagent_type="docs-vision", description="Complete workflow", prompt=...) ``` ## Success Criteria @@ -427,4 +508,7 @@ Edit(file_path="requirements/asyncpg-pooling/tasks.md", ...) ✅ **Tests pass** - `make lint` and `make test` pass ✅ **Performance considered** - SQLglot and mypyc patterns followed ✅ **Workspace updated** - tasks.md and recovery.md current -✅ **Clean handoff** - Next agent (Testing/Docs) can resume easily +✅ **Testing agent invoked** - Tests created and passing +✅ **Docs & Vision invoked** - Documentation, quality gate, knowledge capture, and archive complete +✅ **Spec archived** - Moved to specs/archive/ +✅ **Knowledge captured** - AGENTS.md and guides updated with new patterns diff --git a/.claude/agents/planner.md b/.claude/agents/planner.md index 819e0e23b..dc5d45637 100644 --- a/.claude/agents/planner.md +++ b/.claude/agents/planner.md @@ -14,10 +14,12 @@ Strategic planning agent for SQLSpec development. Creates research-grounded, mul 1. **Research-Grounded Planning** - Consult guides, docs, and best practices before planning 2. **Multi-Session Planning** - Use zen planner for structured, resumable plans 3. **Consensus Verification** - Get multi-model agreement on complex decisions -4. **Session Continuity** - Produce detailed artifacts in `requirements/` workspace +4. **Session Continuity** - Produce detailed artifacts in `specs/active/` workspace ## Planning Workflow +Codex or Gemini CLI can mirror this workflow without using `/plan`. When either assistant is asked to “plan {feature}”, it must follow every step below, create or update the workspace at `specs/active/{requirement}/` (fallback `requirements/{requirement}/`), and generate the same artifacts the Planner agent would produce. Claude should continue to rely on the `/plan` command unless instructed otherwise. + ### Step 1: Understand Requirements ```python @@ -111,10 +113,10 @@ mcp__zen__consensus( ### Step 5: Create Workspace Artifacts -Create requirement folder in `requirements/`: +Create requirement folder in `specs/active/`: ```bash -mkdir -p requirements/{requirement-slug}/{research,tmp} +mkdir -p specs/active/{requirement-slug}/{research,tmp} ``` **Required files:** @@ -230,14 +232,14 @@ After planning complete: 1. **Verify workspace created**: ```bash - ls -la requirements/{requirement-slug}/ + ls -la specs/active/{requirement-slug}/ # Should show: prd.md, tasks.md, research/, tmp/, recovery.md ``` 2. **Notify user**: ``` - Planning complete! Workspace created at `requirements/{requirement-slug}/`. + Planning complete! Workspace created at `specs/active/{requirement-slug}/`. Next: Invoke Expert agent to begin implementation. ``` @@ -290,6 +292,6 @@ mcp__zen__planner( ✅ **Research complete** - All relevant guides consulted ✅ **Plan structured** - Zen planner workflow used ✅ **Decisions verified** - Consensus on complex choices -✅ **Workspace created** - `requirements/{requirement}/` fully populated +✅ **Workspace created** - `specs/active/{requirement}/` fully populated ✅ **Resumable** - recovery.md enables session continuity ✅ **Standards followed** - CLAUDE.md patterns enforced diff --git a/.claude/agents/testing.md b/.claude/agents/testing.md index aa14cd022..595c034aa 100644 --- a/.claude/agents/testing.md +++ b/.claude/agents/testing.md @@ -19,14 +19,16 @@ Comprehensive testing specialist for SQLSpec. Creates pytest-based unit and inte ## Testing Workflow +Codex or Gemini CLI can execute this workflow directly. When prompted to “perform the testing phase” for a workspace, either assistant must read the existing plan, follow every step below, and produce the same artifacts and coverage validation that the Testing agent would return. Claude should continue to use `/test` unless instructed otherwise. + ### Step 1: Read Implementation Understand what needs testing: ```python # Read workspace -Read("requirements/{requirement}/prd.md") -Read("requirements/{requirement}/tasks.md") +Read("specs/active/{requirement}/prd.md") +Read("specs/active/{requirement}/tasks.md") # Read implementation Read("sqlspec/adapters/asyncpg/driver.py") @@ -290,7 +292,7 @@ uv run pytest -n 2 --dist=loadgroup **Mark testing complete:** ```markdown -# In requirements/{requirement}/tasks.md: +# In specs/active/{requirement}/tasks.md: - [x] 3. Adapter-specific code - [x] 4. Testing ← JUST COMPLETED - [ ] 5. Documentation ← HAND OFF TO DOCS & VISION @@ -308,10 +310,12 @@ Tests added: All tests passing ✅ ## Next Steps -Documentation agent should: +Docs & Vision agent (auto-invoked by Expert) should: - Update adapter documentation -- Add usage examples -- Update API reference +- Run quality gate +- Capture new testing patterns in AGENTS.md +- Update docs/guides/ with patterns +- Re-validate and archive ``` ## Test Organization @@ -404,7 +408,7 @@ open htmlcov/index.html assert result is not None ``` -## Handoff to Docs & Vision +## Return to Expert Agent When testing complete: @@ -418,25 +422,28 @@ When testing complete: 2. **Update workspace:** ```markdown + # In specs/active/{requirement}/tasks.md: - [x] 4. Testing - - [ ] 5. Documentation ← HAND OFF TO DOCS & VISION + - [ ] 5. Documentation ← EXPERT WILL AUTO-INVOKE DOCS & VISION ``` -3. **Notify user:** +3. **Return to Expert with summary:** ``` Testing complete! ✅ Tests added: - - [tests/integration/test_adapters/test_asyncpg/test_connection.py](tests/integration/test_adapters/test_asyncpg/test_connection.py) - - [tests/unit/test_core/test_statement.py](tests/unit/test_core/test_statement.py) + - [tests/integration/test_adapters/test_asyncpg/test_connection.py](tests/integration/test_adapters/test_asyncpg/test_connection.py) (5 tests) + - [tests/unit/test_core/test_statement.py](tests/unit/test_core/test_statement.py) (3 tests) Coverage: 87% (target: 80%+) All tests passing: ✅ - Next: Invoke Docs & Vision agent for documentation and final review. + Expert agent will now auto-invoke Docs & Vision for documentation, quality gate, knowledge capture, and archival. ``` +**Note**: This agent is typically invoked automatically by the Expert agent. It returns control to Expert, which then auto-invokes Docs & Vision agent. + ## Tools Available - **Context7** - Library documentation (pytest, pytest-asyncio, pytest-databases) diff --git a/.claude/commands/implement.md b/.claude/commands/implement.md index 5946afdb7..5300dd957 100644 --- a/.claude/commands/implement.md +++ b/.claude/commands/implement.md @@ -2,21 +2,35 @@ Implement the feature from the active workspace. Invoke the Expert agent to: -1. **Read Plan** - Load prd.md, tasks.md, research/plan.md from requirements/{requirement}/ +1. **Read Plan** - Load prd.md, tasks.md, research/plan.md from specs/active/{requirement}/ 2. **Research** - Consult guides and library docs -3. **Implement** - Write clean, type-safe, performant code following CLAUDE.md standards +3. **Implement** - Write clean, type-safe, performant code following AGENTS.md standards 4. **Test** - Run relevant tests to verify implementation -5. **Update** - Mark tasks complete in tasks.md, update recovery.md +5. **Auto-invoke Testing agent** - Create comprehensive test suite (automatic) +6. **Auto-invoke Docs & Vision agent** - Full 5-phase workflow (automatic): + - Phase 1: Update documentation + - Phase 2: Quality gate validation + - Phase 3: **Knowledge capture** (update AGENTS.md + guides) + - Phase 4: **Re-validation** (verify consistency after updates) + - Phase 5: Clean and archive The expert should: -- Follow CLAUDE.md code quality standards (NO hasattr, NO workaround naming, etc.) +- Follow AGENTS.md code quality standards (NO hasattr, NO workaround naming, etc.) - Reference docs/guides/ for patterns - Use zen.debug for complex bugs - Use zen.thinkdeep for architectural decisions - Use zen.analyze for code analysis - Update workspace progress continuously -After implementation, hand off to testing. +**This ONE command handles:** +✅ Implementation +✅ Testing (automatic via Testing agent) +✅ Documentation (automatic via Docs & Vision) +✅ Quality gate (automatic via Docs & Vision) +✅ **Knowledge capture** (automatic via Docs & Vision) +✅ **Re-validation** (automatic via Docs & Vision) +✅ Archival (automatic via Docs & Vision) -Next step: Run `/test` to create comprehensive tests. +**After implementation:** +Feature is complete, tested, documented, patterns captured, and archived! diff --git a/.claude/commands/plan.md b/.claude/commands/plan.md index 1c1212f87..325c7bbd1 100644 --- a/.claude/commands/plan.md +++ b/.claude/commands/plan.md @@ -5,7 +5,7 @@ Invoke the Planner agent to: 1. **Research** - Consult docs/guides/, Context7, and WebSearch 2. **Plan** - Use zen.planner for structured planning 3. **Consensus** - Get multi-model agreement on complex decisions -4. **Workspace** - Create requirements/{requirement-slug}/ with: +4. **Workspace** - Create specs/active/{requirement-slug}/ with: - prd.md (Product Requirements Document) - tasks.md (Implementation checklist) - research/plan.md (Research findings) @@ -19,8 +19,8 @@ The planner should reference: - docs/guides/testing/ for test planning - docs/guides/architecture/ for design patterns - docs/guides/quick-reference/ for common patterns -- CLAUDE.md for code quality standards +- AGENTS.md for code quality standards After planning, the workspace will be ready for implementation. -Next step: Run `/implement` to begin development. +Next step: Run `/implement` to begin development (auto-runs testing, docs, knowledge capture, archive). diff --git a/.claude/commands/review.md b/.claude/commands/review.md index 804d584f8..2f8d3c0ff 100644 --- a/.claude/commands/review.md +++ b/.claude/commands/review.md @@ -1,33 +1,42 @@ -Run documentation, quality gate, and mandatory cleanup. - -Invoke the Docs & Vision agent to run all 3 phases: - -## Phase 1: Documentation - -- Update docs/guides/ as needed -- Update API reference (docs/reference/) -- Add usage examples -- Build docs locally to verify - -## Phase 2: Quality Gate (MANDATORY) - -- Run `make lint` - must pass -- Check for anti-patterns (hasattr, workaround naming, class tests) -- Run full test suite - must pass -- Verify PRD acceptance criteria met - -**Quality gate MUST pass before proceeding to cleanup.** - -## Phase 3: Cleanup (MANDATORY) - -- Remove all requirements/*/tmp/ directories -- Archive completed requirement to requirements/archive/ -- Keep only last 3 active requirements in requirements/ -- Archive planning reports to .claude/reports/archive/ -- Verify workspace is clean - -**Cleanup is MANDATORY - never skip this phase.** - -After review complete, work is ready for commit/PR. - -Final step: Run `make lint && make test` one more time, then commit! +Complete documentation, quality gate, knowledge capture, and archival for the implemented feature. + +Invoke the Docs & Vision agent to run the full 5-phase workflow: + +**Phase 1: Documentation** +- Update API documentation +- Create/update guides in docs/guides/ +- Validate code examples work +- Build documentation without errors + +**Phase 2: Quality Gate** +- Verify all PRD acceptance criteria met +- Verify all tests passing +- Check code standards compliance (AGENTS.md) +- BLOCK if any criteria not met + +**Phase 3: Knowledge Capture (NEW!)** +- Analyze implementation for new patterns +- Extract best practices and conventions +- **Update AGENTS.md with new patterns** +- **Update relevant guides in docs/guides/** +- Document patterns with working examples + +**Phase 4: Re-validation (NEW!)** +- Re-run tests after documentation updates +- Rebuild documentation to verify no errors +- Check pattern consistency across project +- Verify no breaking changes introduced +- BLOCK if re-validation fails + +**Phase 5: Cleanup & Archive** +- Remove all tmp/ files +- Move specs/active/{requirement} to specs/archive/ +- Generate completion report + +**Note:** This command is typically not needed manually because `/implement` automatically invokes Docs & Vision. Use this only if you need to: +- Re-run validation after manual changes +- Regenerate documentation +- Force re-archival +- Update AGENTS.md with new patterns after manual implementation + +After review, feature is documented, validated, patterns captured, and archived! diff --git a/.claude/commands/test.md b/.claude/commands/test.md index aad5b7f36..fd11551a6 100644 --- a/.claude/commands/test.md +++ b/.claude/commands/test.md @@ -2,21 +2,24 @@ Create comprehensive tests for the implemented feature. Invoke the Testing agent to: -1. **Read Implementation** - Understand what needs testing -2. **Consult Guide** - Reference docs/guides/testing/testing.md -3. **Create Unit Tests** - Test individual components (tests/unit/) -4. **Create Integration Tests** - Test with real databases (tests/integration/) -5. **Test Edge Cases** - Empty inputs, None values, errors, concurrency -6. **Verify Coverage** - Ensure 80%+ for adapters, 90%+ for core -7. **Run Tests** - Verify all tests pass +1. **Read Implementation** - Load prd.md, recovery.md from specs/active/{requirement}/ +2. **Create Unit Tests** - Test individual components in isolation +3. **Create Integration Tests** - Test with real database connections +4. **Test Edge Cases** - Empty inputs, None values, errors, concurrency +5. **Validate Coverage** - Ensure >80% coverage for adapters, >90% for core +6. **Update Workspace** - Mark test tasks complete The testing agent should: -- Use function-based tests (def test_something():) -- Mark tests appropriately (@pytest.mark.asyncio, @pytest.mark.postgres, etc.) -- Use pytest-databases fixtures (postgres_url, oracle_url, etc.) -- Test both success and error paths -- Update workspace when complete -After testing, hand off to documentation and quality gate. +- Use function-based tests (NO class-based tests) +- Follow pytest patterns from docs/guides/testing/testing.md +- Test all affected database adapters +- Use pytest markers (@pytest.mark.postgres, etc.) +- Verify all tests pass before returning -Next step: Run `/review` for documentation, quality gate, and cleanup. +**Note:** This command is typically not needed manually because `/implement` automatically invokes the Testing agent. Use this only if you need to: +- Re-create tests after manual changes +- Add additional test coverage +- Debug test failures + +After testing, Expert agent will auto-invoke Docs & Vision for documentation, quality gate, knowledge capture, and archival. diff --git a/.gitignore b/.gitignore index 036ca1f33..35cf5017d 100644 --- a/.gitignore +++ b/.gitignore @@ -56,6 +56,10 @@ docs-build !.claude/agents !.claude/AGENTS.md !.claude/README.md +specs/active/* +!specs/active/.gitkeep +specs/archive/* +!specs/archive/.gitkeep requirements/archive/* !requirements/archive/.gitkeep requirements/* diff --git a/AGENTS.md b/AGENTS.md index fb46b8395..551492aff 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1034,3 +1034,231 @@ class AdapterDriverFeatures(TypedDict): - **Type Handler Pattern** (above): Implementation details for type handlers used with `driver_features` - **Optional Dependency Handling**: See `sqlspec.typing` for detection constants - **Testing Standards**: See Testing Strategy section for general testing requirements + +## Agent Workflow Coordination + +### Automated Multi-Agent Workflow + +SQLSpec uses a coordinated multi-agent system where the Expert agent orchestrates the complete development lifecycle: + +``` +User runs: /implement {feature-name} + +┌─────────────────────────────────────────────────────────────┐ +│ EXPERT AGENT │ +│ │ +│ 1. Read Plan & Research (from specs/active/{feature}/) │ +│ 2. Implement Feature (following AGENTS.md standards) │ +│ 3. Self-Test & Verify │ +│ 4. ──► Auto-Invoke Testing Agent (subagent) │ +│ │ │ +│ ├─► Create unit tests │ +│ ├─► Create integration tests (all adapters) │ +│ ├─► Test edge cases │ +│ └─► Verify coverage & all tests pass │ +│ 5. ──► Auto-Invoke Docs & Vision Agent (subagent) │ +│ │ │ +│ ├─► Phase 1: Update documentation │ +│ ├─► Phase 2: Quality gate validation │ +│ ├─► Phase 3: Knowledge capture (AGENTS.md+guides) │ +│ ├─► Phase 4: Re-validate after updates │ +│ ├─► Phase 5: Clean tmp/ and archive │ +│ └─► Generate completion report │ +│ 6. Return Complete Summary │ +└─────────────────────────────────────────────────────────────┘ + +Result: Feature implemented, tested, documented, archived - all automatically! +``` + +### Codex-Oriented Workflow Usage + +Codex can execute the same lifecycle without invoking Claude slash commands. Use the prompts below to engage Codex directly while keeping the workflow artifacts identical. + +- **General Rule**: Tell Codex which phase to emulate (`plan`, `implement`, `test`, `review`) and point to the active workspace root (`specs/active/{slug}/` preferred). Codex will create the folder if it does not exist. +- **Codex `/plan` Equivalent**: Ask "Codex: run planning for {feature}" and provide any context. Codex must (1) research via docs/guides/ as outlined in `.claude/agents/planner.md`, (2) write or update `prd.md`, `tasks.md`, `research/plan.md`, `recovery.md`, and (3) ensure `tmp/` exists. Planning output follows the same structure the Planner agent would create. +- **Codex `/implement` Equivalent**: Ask Codex to "execute implementation phase for {workspace}". Codex then reads the workspace, consults guides, writes code under `sqlspec/`, updates tasks, and runs local checks exactly as described in `.claude/agents/expert.md`. When the plan calls for sub-agents, Codex continues by emulating the Testing and Docs & Vision phases in order. +- **Codex `/test` Equivalent**: Request "Codex: perform testing phase for {workspace}". Codex creates or updates pytest suites, ensures coverage thresholds, and records progress in `tasks.md`, mirroring `.claude/agents/testing.md`. +- **Codex `/review` Equivalent**: Request "Codex: run docs, quality gate, and cleanup for {workspace}". Codex completes the five Docs & Vision phases—documentation, quality gate, knowledge capture (including AGENTS.md and guides updates), re-validation, and workspace archival. +- **Knowledge Capture Reminder**: Whenever Codex finishes implementation or review work, it must update this AGENTS.md and any relevant guides with new patterns so Claude and other assistants inherit the learnings. + +### Gemini CLI Workflow Usage + +Gemini CLI can execute the same lifecycle. Direct Gemini to the desired phase and reference the active workspace so it mirrors the Planner, Expert, Testing, and Docs & Vision agents. + +- **General Rule**: Specify the phase (`plan`, `implement`, `test`, `review`) and point Gemini to `specs/active/{slug}/` (fallback `requirements/{slug}/`). Gemini should create the workspace if it is missing and follow `.claude/agents/{agent}.md` for detailed steps. +- **Gemini `/plan` Equivalent**: Prompt "Gemini: plan {feature} using AGENTS.md." Gemini reads the guides, writes `prd.md`, `tasks.md`, `research/plan.md`, `recovery.md`, and ensures a `tmp/` directory exists. +- **Gemini `/implement` Equivalent**: Prompt "Gemini: run implementation phase for {workspace}." Gemini reads the workspace artifacts, implements code per `.claude/agents/expert.md`, and continues by emulating the Testing and Docs & Vision workflows in sequence. +- **Gemini `/test` Equivalent**: Prompt "Gemini: execute testing phase for {workspace}." Gemini creates or updates pytest suites, verifies coverage targets, and records progress in `tasks.md` exactly like the Testing agent. +- **Gemini `/review` Equivalent**: Prompt "Gemini: perform docs, quality gate, and cleanup for {workspace}." Gemini completes all five Docs & Vision phases, including knowledge capture updates to AGENTS.md and guides, followed by archival. +- **Prompt Templates**: If using Gemini CLI prompt files, include a directive to consult the relevant agent guide plus this section so each invocation stays aligned. + +### Claude Workflow Usage + +Claude already maps to these phases through the slash commands defined in `.claude/commands/`. Use the commands or free-form prompts—the agent guides remain the source of truth. + +- **Default Flow**: `/plan`, `/implement`, `/test`, `/review` trigger the Planner, Expert, Testing, and Docs & Vision workflows automatically. +- **Manual Prompts**: When not using slash commands, instruct Claude which phase to run and provide the workspace path so it follows the same sequence. +- **Knowledge Capture Expectation**: Claude must update AGENTS.md and guides during the Docs & Vision phase before archiving, regardless of invocation style. + +**Key Workflow Principles**: + +- **Single Command**: `/implement` handles entire lifecycle +- **No Manual Steps**: Testing, docs, and archival automatic +- **Knowledge Preservation**: Patterns captured in AGENTS.md and guides +- **Quality Assurance**: Multi-phase validation before completion +- **Session Resumability**: All work tracked in specs/active/{feature}/ + +### Knowledge Capture Process + +After every feature implementation, the Docs & Vision agent **must** extract and preserve new patterns: + +#### Step 1: Analyze Implementation + +Review what was built for reusable patterns: + +- **New Patterns**: Novel approaches to common problems +- **Best Practices**: Techniques that worked particularly well +- **Conventions**: Naming, structure, or organization patterns +- **Type Handling**: New type conversion or validation approaches +- **Testing Patterns**: Effective test strategies +- **Performance Techniques**: Optimization discoveries + +#### Step 2: Update AGENTS.md + +Add patterns to relevant sections in this file: + +- **Code Quality Standards** - New coding patterns +- **Testing Strategy** - New test approaches +- **Performance Optimizations** - New optimization techniques +- **Database Adapter Implementation** - Adapter-specific patterns +- **driver_features Pattern** - New feature configurations + +Example addition: + +```python +# In Docs & Vision agent +Edit( + file_path="AGENTS.md", + old_string="### Compliance Table", + new_string="""### New Pattern: Session Callbacks for Type Handlers + +When implementing optional type handlers: + +```python +class AdapterConfig(AsyncDatabaseConfig): + async def _create_pool(self): + config = dict(self.pool_config) + if self.driver_features.get("enable_feature", False): + config["session_callback"] = self._init_connection + return await create_pool(**config) + + async def _init_connection(self, connection): + if self.driver_features.get("enable_feature", False): + from ._feature_handlers import register_handlers + register_handlers(connection) +``` + +### Compliance Table""" + +) + +``` + +#### Step 3: Update docs/guides/ + +Enhance relevant guides with new patterns: + +- `docs/guides/adapters/{adapter}.md` - Adapter-specific patterns +- `docs/guides/testing/testing.md` - Testing patterns +- `docs/guides/performance/` - Performance techniques +- `docs/guides/architecture/` - Architectural patterns + +#### Step 4: Validate Examples + +Ensure all new patterns have working code examples that execute successfully. + +## Re-validation Protocol + +After updating AGENTS.md or guides, the Docs & Vision agent **must** re-validate to ensure consistency: + +### Step 1: Re-run Tests + +```bash +uv run pytest -n 2 --dist=loadgroup +``` + +**All tests must still pass** after documentation updates. + +### Step 2: Rebuild Documentation + +```bash +make docs +``` + +**Documentation must build without new errors** after updates. + +### Step 3: Verify Pattern Consistency + +Manually check that: + +- New patterns align with existing standards +- No contradictory advice introduced +- Examples follow project conventions +- Terminology is consistent + +### Step 4: Check for Breaking Changes + +Verify no unintended breaking changes in documentation updates. + +### Step 5: Block if Re-validation Fails + +**DO NOT archive the spec** if re-validation fails: + +- Fix issues introduced by documentation updates +- Re-run re-validation +- Only proceed to archival when all checks pass + +## Workspace Management + +### Folder Structure + +``` +specs/ +├── active/ # Active work (gitignored) +│ ├── {feature-name}/ +│ │ ├── prd.md # Product Requirements Document +│ │ ├── tasks.md # Phase-by-phase checklist +│ │ ├── recovery.md # Session resume guide +│ │ ├── research/ # Research findings +│ │ └── tmp/ # Temporary files (cleaned by Docs & Vision) +│ └── .gitkeep +├── archive/ # Completed work (gitignored) +│ └── {completed-feature}/ +├── template-spec/ # Template structure (committed) +│ ├── prd.md +│ ├── tasks.md +│ ├── recovery.md +│ ├── README.md +│ ├── research/.gitkeep +│ └── tmp/.gitkeep +└── README.md +``` + +### Lifecycle + +1. **Planning**: Planner creates `specs/active/{feature}/` +2. **Implementation**: Expert implements, auto-invokes Testing and Docs & Vision +3. **Testing**: Testing agent creates comprehensive test suite (automatic) +4. **Documentation**: Docs & Vision updates docs (automatic) +5. **Knowledge Capture**: Docs & Vision extracts patterns (automatic) +6. **Re-validation**: Docs & Vision verifies consistency (automatic) +7. **Archive**: Docs & Vision moves to `specs/archive/{feature}/` (automatic) + +### Session Resumability + +Any agent can resume work by reading: + +1. `specs/active/{feature}/recovery.md` - Current status and next steps +2. `specs/active/{feature}/tasks.md` - What's complete +3. `specs/active/{feature}/prd.md` - Full requirements +4. `specs/active/{feature}/research/` - Findings and analysis diff --git a/requirements/README.md b/requirements/README.md deleted file mode 100644 index 8268614f1..000000000 --- a/requirements/README.md +++ /dev/null @@ -1,138 +0,0 @@ -# requirements/ Workspace - -Active requirement planning and development workspace for AI coding agents. - -## Structure - -Each requirement gets a dedicated folder: - -``` -requirements/ -├── {requirement-slug}/ # Active requirement -│ ├── prd.md # Product Requirements Document -│ ├── tasks.md # Implementation checklist -│ ├── recovery.md # Session resume instructions -│ ├── research/ # Research findings, plans -│ │ └── plan.md # Detailed planning output -│ └── tmp/ # Temporary files (cleaned by Docs & Vision) -├── archive/ # Completed requirements -│ └── {old-requirement}/ # Archived when complete -└── README.md # This file -``` - -## Workflow - -### 1. Planning (`/plan`) - -Planner agent creates requirement folder: - -```bash -requirements/vector-search/ -├── prd.md # Created by Planner -├── tasks.md # Created by Planner -├── research/ # Created by Planner -│ └── plan.md -├── tmp/ # Created by Planner -└── recovery.md # Created by Planner -``` - -### 2. Implementation (`/implement`) - -Expert agent: - -- Reads prd.md, tasks.md, research/plan.md -- Implements feature -- Updates tasks.md (marks items complete) -- Updates recovery.md (current status) -- May create tmp files (cleaned later) - -### 3. Testing (`/test`) - -Testing agent: - -- Reads implementation -- Creates tests -- Updates tasks.md -- Updates recovery.md - -### 4. Review (`/review`) - -Docs & Vision agent: - -- Writes documentation -- Runs quality gate -- **MANDATORY CLEANUP**: - - Removes all tmp/ directories - - Archives completed requirement - - Keeps only last 3 active requirements - -## Cleanup Protocol - -**MANDATORY after every `/review`:** - -1. **Remove tmp/ directories:** - - ```bash - find requirements/*/tmp -type d -exec rm -rf {} + - ``` - -2. **Archive completed work:** - - ```bash - mv requirements/{requirement} requirements/archive/{requirement} - ``` - -3. **Keep only last 3 active:** - - ```bash - # Move oldest requirements to archive if more than 3 active - ``` - -## Session Continuity - -To resume work across sessions: - -```python -# Read recovery.md to understand current state -Read("requirements/{requirement}/recovery.md") - -# Check tasks.md for what's complete -Read("requirements/{requirement}/tasks.md") - -# Review PRD for full context -Read("requirements/{requirement}/prd.md") - -# Review research findings -Read("requirements/{requirement}/research/plan.md") -``` - -## Active Requirements Limit - -**Keep only 3 active requirements** in requirements/ root: - -- Prevents workspace clutter -- Forces completion of old work -- Maintains focus on current priorities -- Older requirements auto-archived - -## Archive Management - -Completed requirements in `requirements/archive/` are: - -- Preserved for reference -- Searchable for patterns -- Available for recovery if needed -- Never deleted (historical record) - -## Usage with AI Agents - -All agents (Claude, Gemini, Codex) use this workspace: - -- **Planner** creates the structure -- **Expert** implements and updates -- **Testing** adds test tracking -- **Docs & Vision** enforces cleanup - -## Example - -See `requirements/example-feature/` for reference structure. diff --git a/requirements/example-feature/prd.md b/requirements/example-feature/prd.md deleted file mode 100644 index 02063fb90..000000000 --- a/requirements/example-feature/prd.md +++ /dev/null @@ -1,78 +0,0 @@ -# Feature: Example Feature - -This is an example Product Requirements Document (PRD). - -## Overview - -Brief description of the feature, why it's needed, and what problem it solves. - -## Acceptance Criteria - -- [ ] Criterion 1 - Feature works as described -- [ ] Criterion 2 - Edge cases handled (empty, None, errors) -- [ ] Criterion 3 - Performance acceptable (no regressions) -- [ ] Criterion 4 - Tests comprehensive (80%+ coverage) -- [ ] Criterion 5 - Documentation complete - -## Technical Design - -### Affected Components - -- `sqlspec/core/result.py` - Add new method -- `sqlspec/adapters/asyncpg/driver.py` - Implement adapter-specific logic - -### Database Adapters Affected - -- AsyncPG (PostgreSQL) -- Oracle -- DuckDB - -### API Changes - -```python -# New method on SQLResult -class SQLResult: - def to_format(self, format: str) -> Any: - """Convert result to specified format. - - Args: - format: Output format (arrow, pandas, polars, dict) - - Returns: - Result in specified format. - """ - pass -``` - -## Dependencies - -- Requires pyarrow for Arrow format -- Requires pandas for DataFrame format -- Requires polars for Polars format - -## Testing Strategy - -### Unit Tests -- Test format conversion logic -- Test error handling (invalid format) -- Test empty result sets - -### Integration Tests -- Test with real asyncpg connection -- Test with real oracle connection -- Test with real duckdb connection -- Test large result sets (10k+ rows) - -## Migration Notes - -Non-breaking change - new functionality only. - -## Performance Considerations - -- Arrow format should be zero-copy when possible -- Pandas/Polars conversion may allocate new memory -- Large result sets (1M+ rows) should stream - -## Security Considerations - -None - read-only operation. diff --git a/requirements/example-feature/recovery.md b/requirements/example-feature/recovery.md deleted file mode 100644 index d4f44eac9..000000000 --- a/requirements/example-feature/recovery.md +++ /dev/null @@ -1,43 +0,0 @@ -# Recovery: Example Feature - -## To Resume Work - -1. Read [prd.md](prd.md) for full context -2. Check [tasks.md](tasks.md) for progress -3. Review [research/plan.md](research/plan.md) for implementation details -4. Check current status below - -## Current Status - -**Status:** Example (not started) -**Last updated:** 2025-10-09 -**Current phase:** Planning - -## Progress Summary - -Nothing started yet - this is an example. - -## Files Modified - -None. - -## Tests Added - -None. - -## Next Steps - -This is an example requirement folder. - -Real requirements created by `/plan` will have: -- Complete PRD with acceptance criteria -- Detailed research findings in research/plan.md -- Clear next steps for resuming work -- Current status and progress tracking - -## How to Use This Example - -1. Run `/plan {your-feature-description}` to create real requirement -2. Planner will create folder structure like this one -3. Follow workflow: plan → implement → test → review -4. Docs & Vision will archive when complete diff --git a/requirements/example-feature/research/plan.md b/requirements/example-feature/research/plan.md deleted file mode 100644 index 6d67e5569..000000000 --- a/requirements/example-feature/research/plan.md +++ /dev/null @@ -1,117 +0,0 @@ -# Planning: Example Feature - -This is an example planning document created by the Planner agent. - -## Research Findings - -### Static Guides Consulted - -- `docs/guides/performance/sqlglot-best-practices.md` - SQL optimization patterns -- `docs/guides/architecture/data-flow.md` - How data flows through SQLSpec -- `docs/guides/adapters/asyncpg.md` - PostgreSQL patterns - -### Context7 Documentation - -- `/apache/arrow` - PyArrow conversion best practices -- `/pandas-dev/pandas` - DataFrame creation from Arrow -- `/pola-rs/polars` - Polars DataFrame creation - -### WebSearch Findings - -- Arrow zero-copy best practices (2025) -- Pandas performance optimization (2025) -- Polars vs Pandas benchmarks - -## Implementation Plan - -### Phase 1: Core Implementation - -1. Add `to_format()` method to `sqlspec/core/result.py`: - ```python - def to_format(self, format: str) -> Any: - """Convert result to specified format.""" - if format == "arrow": - return self._to_arrow() - elif format == "pandas": - return self._to_pandas() - elif format == "polars": - return self._to_polars() - elif format == "dict": - return self.as_dicts() - else: - raise ValueError(f"Unknown format: {format}") - ``` - -2. Implement format-specific converters: - - `_to_arrow()` - Use existing Arrow support - - `_to_pandas()` - Convert via Arrow (zero-copy) - - `_to_polars()` - Convert via Arrow (zero-copy) - -### Phase 2: Adapter-Specific Code - -No adapter-specific code needed - core implementation works for all adapters. - -### Phase 3: Testing - -See tasks.md for comprehensive test list. - -### Phase 4: Documentation - -Add examples to quick reference guide. - -## Architecture Decisions - -### Decision: Use Arrow as Intermediate Format - -**Consensus:** Gemini-2.5-Pro ✅, GPT-5 ✅ - -**Rationale:** -- Arrow is already supported -- Zero-copy conversion to Pandas/Polars -- Efficient for large result sets -- Industry standard - -### Decision: Error Handling Strategy - -Raise `ValueError` for unsupported formats rather than returning None or empty results. - -**Rationale:** -- Fail fast principle -- Clear error messages -- User can catch and handle - -## Performance Considerations - -### Zero-Copy Conversions - -Arrow → Pandas and Arrow → Polars can be zero-copy: - -```python -# Zero-copy when possible -arrow_table = result.to_arrow() -pandas_df = arrow_table.to_pandas(zero_copy_only=True) -``` - -### Large Result Sets - -For 1M+ row results: -- Use streaming when possible -- Consider chunked conversion -- Monitor memory usage - -## Edge Cases - -- Empty result set → Return empty Arrow table / DataFrame -- Single row → Should work same as multi-row -- Single column → Should work -- NULL values → Arrow handles natively - -## Testing Checklist - -See tasks.md for full testing checklist. - -## Next Steps - -1. Expert agent implements core functionality -2. Testing agent creates comprehensive tests -3. Docs & Vision documents and validates diff --git a/requirements/example-feature/tasks.md b/requirements/example-feature/tasks.md deleted file mode 100644 index 8c92c62c7..000000000 --- a/requirements/example-feature/tasks.md +++ /dev/null @@ -1,59 +0,0 @@ -# Tasks: Example Feature - -## Workflow Stages - -- [ ] 1. Research & Planning (Planner agent) -- [ ] 2. Core Implementation (Expert agent) -- [ ] 3. Adapter-Specific Code (Expert agent) -- [ ] 4. Testing (Testing agent) -- [ ] 5. Documentation (Docs & Vision agent) -- [ ] 6. Quality Gate (Docs & Vision agent) -- [ ] 7. Cleanup (Docs & Vision agent) - -## Detailed Tasks - -### Planning - -- [ ] Research Arrow, Pandas, Polars conversion patterns -- [ ] Consult docs/guides/performance/ for optimization -- [ ] Create detailed implementation plan -- [ ] Get consensus on API design (if needed) - -### Implementation - -- [ ] Add `to_format()` method to SQLResult -- [ ] Implement Arrow format conversion -- [ ] Implement Pandas format conversion -- [ ] Implement Polars format conversion -- [ ] Add error handling for invalid formats -- [ ] Optimize for zero-copy when possible - -### Testing - -- [ ] Unit test: format conversion logic -- [ ] Unit test: error handling -- [ ] Unit test: empty results -- [ ] Integration test: AsyncPG with Arrow format -- [ ] Integration test: Oracle with Pandas format -- [ ] Integration test: DuckDB with Polars format -- [ ] Integration test: Large result set (10k rows) - -### Documentation - -- [ ] Update docs/guides/quick-reference/quick-reference.md -- [ ] Update API reference (docs/reference/) -- [ ] Add usage examples -- [ ] Build docs locally - -### Quality Gate - -- [ ] Run `make lint` - must pass -- [ ] Check for anti-patterns -- [ ] Run full test suite -- [ ] Verify PRD acceptance criteria - -### Cleanup - -- [ ] Remove tmp/ directory -- [ ] Archive requirement to requirements/archive/ -- [ ] Archive planning reports diff --git a/specs/README.md b/specs/README.md new file mode 100644 index 000000000..6d3f7b8a5 --- /dev/null +++ b/specs/README.md @@ -0,0 +1,175 @@ +# specs/ Workspace + +Active specification planning and development workspace for AI coding agents working on SQLSpec. + +## Structure + +Each specification gets a dedicated folder: + +``` +specs/ +├── active/ # Active work (gitignored) +│ ├── {spec-slug}/ +│ │ ├── prd.md # Product Requirements Document +│ │ ├── tasks.md # Implementation checklist +│ │ ├── recovery.md # Session resume instructions +│ │ ├── research/ # Research findings, plans +│ │ │ └── plan.md # Detailed planning output +│ │ └── tmp/ # Temporary files (cleaned by Docs & Vision) +│ └── .gitkeep +├── archive/ # Completed work (gitignored) +│ └── {archived-spec}/ +├── template-spec/ # Template structure (committed) +│ ├── prd.md +│ ├── tasks.md +│ ├── recovery.md +│ ├── README.md +│ ├── research/.gitkeep +│ └── tmp/.gitkeep +└── README.md # This file +``` + +## Workflow + +### 1. Planning (`/plan`) + +Planner agent creates spec folder: + +```bash +specs/active/vector-search/ +├── prd.md # Created by Planner +├── tasks.md # Created by Planner +├── recovery.md # Created by Planner +├── research/ # Created by Planner +│ └── plan.md +└── tmp/ # Created by Planner +``` + +### 2. Implementation (`/implement`) + +Expert agent orchestrates the **entire workflow automatically**: + +- Reads prd.md, tasks.md, research/plan.md +- Implements feature following AGENTS.md standards +- Updates tasks.md (marks items complete) +- Updates recovery.md (current status) +- **Auto-invokes Testing agent** (creates comprehensive tests) +- **Auto-invokes Docs & Vision agent** (5-phase workflow): + - Phase 1: Documentation + - Phase 2: Quality gate + - Phase 3: **Knowledge capture** (updates AGENTS.md + guides) + - Phase 4: **Re-validation** (verifies consistency) + - Phase 5: Cleanup and archive + +**Result**: Feature implemented, tested, documented, patterns captured, and archived - all from ONE command! + +### 3. Automatic Phases (No Manual Steps) + +When you run `/implement`, everything happens automatically: + +``` +┌─────────────────────────────────────────────────────────────┐ +│ EXPERT AGENT │ +│ │ +│ 1. Read Plan & Research │ +│ 2. Implement Feature │ +│ 3. Self-Test & Verify │ +│ 4. ──► Auto-Invoke Testing Agent │ +│ └─► Creates comprehensive test suite │ +│ 5. ──► Auto-Invoke Docs & Vision Agent │ +│ ├─► Updates documentation │ +│ ├─► Runs quality gate │ +│ ├─► Captures patterns in AGENTS.md │ +│ ├─► Re-validates consistency │ +│ └─► Archives to specs/archive/ │ +│ 6. Returns Complete Summary │ +└─────────────────────────────────────────────────────────────┘ +``` + +## Cleanup Protocol + +**MANDATORY after every `/implement`:** + +The Docs & Vision agent automatically: + +1. **Removes tmp/ directories:** + ```bash + find specs/active/*/tmp -type d -exec rm -rf {} + + ``` + +2. **Archives completed work:** + ```bash + mv specs/active/{spec} specs/archive/{spec} + ``` + +3. **Updates knowledge base:** + - Extracts patterns from implementation + - Updates AGENTS.md with new patterns + - Updates docs/guides/ with new techniques + - Re-validates after updates + +## Session Continuity + +To resume work across sessions: + +```python +# Read recovery.md to understand current state +Read("specs/active/{spec}/recovery.md") + +# Check tasks.md for what's complete +Read("specs/active/{spec}/tasks.md") + +# Review PRD for full context +Read("specs/active/{spec}/prd.md") + +# Review research findings +Read("specs/active/{spec}/research/plan.md") +``` + +## Example Usage + +```bash +# 1. Plan a new feature +/plan Add connection pooling for DuckDB + +# 2. Implement (handles EVERYTHING automatically) +/implement duckdb-connection-pooling +# → Implementation complete +# → Tests created and passing +# → Documentation updated +# → Patterns captured in AGENTS.md +# → Spec archived to specs/archive/ + +# That's it! Feature is complete. +``` + +## Key Features + +✅ **Automated Workflow** - One command (`/implement`) handles entire lifecycle +✅ **Knowledge Preservation** - Patterns captured in AGENTS.md and guides +✅ **Quality Assurance** - Multi-phase validation before completion +✅ **Session Resumability** - recovery.md enables cross-session work +✅ **Clean Workspace** - Automatic cleanup and archival +✅ **No Manual Steps** - Testing, docs, and knowledge capture all automatic + +## Archive Management + +Completed specs in `specs/archive/` are: + +- Preserved for reference +- Searchable for patterns +- Available for recovery if needed +- Never deleted (historical record) + +## Usage with AI Agents + +All agents (Claude, Gemini, Codex) use this workspace: + +- **Planner** creates the structure +- **Expert** implements and orchestrates workflow +- **Testing** adds comprehensive tests (auto-invoked) +- **Docs & Vision** handles docs, QA, knowledge capture, and cleanup (auto-invoked) + +## Reference + +See `specs/template-spec/` for the complete template structure with examples. diff --git a/specs/active/.gitkeep b/specs/active/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/specs/archive/.gitkeep b/specs/archive/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/specs/template-spec/README.md b/specs/template-spec/README.md new file mode 100644 index 000000000..1d2b549fd --- /dev/null +++ b/specs/template-spec/README.md @@ -0,0 +1,70 @@ +# {Feature Name} + +This workspace contains the planning, implementation, testing, and documentation for the {Feature Name} feature in SQLSpec. + +## Quick Links + +- [PRD (Product Requirements Document)](prd.md) +- [Tasks Checklist](tasks.md) +- [Recovery Guide](recovery.md) +- [Research Findings](research/) +- [SQLSpec Standards](../../AGENTS.md) + +## Status + +**Current Phase**: {Phase} +**Status**: {In Progress / Complete} +**Last Updated**: {Date} + +## Workflow + +This spec follows the automated SQLSpec workflow: + +1. **Plan** (`/plan`) - Planner agent creates this workspace +2. **Implement** (`/implement`) - Expert agent: + - Implements feature following AGENTS.md standards + - Auto-invokes Testing agent (creates comprehensive tests) + - Auto-invokes Docs & Vision agent (docs, QA, knowledge capture, archive) +3. **Complete** - Everything done automatically! + +## Files + +- `prd.md` - Full requirements and acceptance criteria +- `tasks.md` - Phase-by-phase checklist +- `recovery.md` - Resume guide for any agent +- `research/` - Expert findings and analysis +- `tmp/` - Temporary files (cleaned by Docs & Vision) + +## Automated Phases + +When you run `/implement`, the Expert agent automatically handles: + +✅ Implementation (following SQLSpec patterns) +✅ Testing (via Testing agent - all adapters) +✅ Documentation (via Docs & Vision agent) +✅ Quality Gate (via Docs & Vision agent) +✅ **Knowledge Capture** (patterns added to AGENTS.md) +✅ **Re-validation** (after knowledge updates) +✅ Archival (moved to specs/archive/) + +No manual steps required! + +## SQLSpec-Specific Context + +**Key Standards** (from AGENTS.md): +- Type hints: Stringified for non-builtins, NO `from __future__ import annotations` +- Tests: Function-based ONLY (not class-based) +- Imports: No nested imports unless preventing circular imports +- Patterns: Protocol-based, not inheritance +- Documentation: Google-style docstrings + +**Testing Against**: +- Multiple database adapters (AsyncPG, Oracle, DuckDB, etc.) +- Real databases via pytest-databases +- Mypyc compilation compatibility + +**Resources**: +- [Architecture Guide](../../docs/guides/architecture/architecture.md) +- [Testing Guide](../../docs/guides/testing/testing.md) +- [Adapter Patterns](../../docs/guides/adapters/) +- [AGENTS.md](../../AGENTS.md) - **READ THIS FIRST** diff --git a/specs/template-spec/prd.md b/specs/template-spec/prd.md new file mode 100644 index 000000000..b03247521 --- /dev/null +++ b/specs/template-spec/prd.md @@ -0,0 +1,158 @@ +# Feature: {Feature Name} + +## Overview + +{1-2 paragraph description of the feature and its value} + +## Problem Statement + +{What problem does this solve? What pain points does it address?} + +## Goals + +- Primary goal: {Main objective} +- Secondary goals: {Additional objectives} + +## Target Users + +- **{User Type 1}**: {How they benefit} +- **{User Type 2}**: {How they benefit} + +## Technical Scope + +### Technology Stack + +**SQLSpec Stack**: +- Language: Python 3.10+ +- Framework: Litestar (web integration) +- Databases: PostgreSQL, Oracle, DuckDB, MySQL, SQLite, BigQuery +- Testing: pytest with pytest-databases +- Build: uv + make + +### Implementation Details + +**Affected Components**: +- `sqlspec/{component}/` - {Description} + +**Database Adapters Affected**: +- AsyncPG (PostgreSQL) +- Oracle (oracledb) +- DuckDB +- {Others as needed} + +## Acceptance Criteria + +### Functional Requirements +- [ ] Feature works as specified +- [ ] All integrations functional +- [ ] Backward compatible +- [ ] Performance acceptable + +### Technical Requirements +- [ ] Code follows project standards (see AGENTS.md) +- [ ] Tests comprehensive and passing +- [ ] Error handling proper +- [ ] Documentation complete +- [ ] Type hints stringified for non-builtins +- [ ] No `from __future__ import annotations` + +### Testing Requirements +- [ ] Unit tests passing +- [ ] Integration tests passing (all affected adapters) +- [ ] Edge cases covered (empty, None, errors, boundaries) +- [ ] Coverage targets met (>80%) +- [ ] Tests are function-based (NOT class-based) + +## Implementation Phases + +### Phase 1: Planning & Research ✅ +- [x] 1.1 Create requirement workspace +- [x] 1.2 Write comprehensive PRD +- [x] 1.3 Create task breakdown +- [x] 1.4 Identify research questions + +### Phase 2: Expert Research +- [ ] 2.1 Research implementation patterns +- [ ] 2.2 Review adapter integration patterns +- [ ] 2.3 Analyze performance implications +- [ ] 2.4 Check mypyc compilation compatibility +- [ ] 2.5 Document findings in `research/` + +### Phase 3: Core Implementation (Expert) +- [ ] 3.1 Implement core functionality +- [ ] 3.2 Add adapter-specific logic +- [ ] 3.3 Update type definitions +- [ ] 3.4 Handle edge cases +- [ ] 3.5 Follow AGENTS.md standards + +### Phase 4: Integration (Expert) +- [ ] 4.1 Update affected adapters +- [ ] 4.2 Add necessary mixins +- [ ] 4.3 Update configuration +- [ ] 4.4 Test integrations + +### Phase 5: Testing (Automatic via Expert) +- [ ] 5.1 Create unit tests +- [ ] 5.2 Create integration tests (all adapters) +- [ ] 5.3 Test edge cases +- [ ] 5.4 Performance validation +- [ ] 5.5 Achieve coverage targets + +### Phase 6: Documentation (Automatic via Expert) +- [ ] 6.1 Update API documentation +- [ ] 6.2 Create/update guide in docs/guides/ +- [ ] 6.3 Add code examples +- [ ] 6.4 Update CHANGELOG + +### Phase 7: Knowledge Capture & Archive (Automatic via Expert) +- [ ] 7.1 Extract new patterns +- [ ] 7.2 Update AGENTS.md +- [ ] 7.3 Update relevant guides +- [ ] 7.4 Re-validate +- [ ] 7.5 Clean and archive + +## Dependencies + +**Internal Dependencies**: +- {List SQLSpec components this depends on} + +**External Dependencies**: +- {New packages needed? Update pyproject.toml} + +## Risks & Mitigations + +### Risk 1: {Risk Description} +- **Mitigation**: {How to mitigate} + +### Risk 2: Mypyc Compilation Compatibility +- **Mitigation**: Avoid `from __future__ import annotations`, use stringified types + +## Research Questions for Expert + +1. {Question about implementation approach} +2. {Question about adapter-specific handling} +3. {Question about performance optimization} +4. {Question about mypyc compatibility} + +## Success Metrics + +- Feature functional and tested +- Tests passing with >80% coverage +- Documentation complete +- No performance regression +- Zero breaking changes +- Mypyc compilation succeeds +- New patterns captured in AGENTS.md + +## References + +**Similar Features**: +- {Link to similar code in SQLSpec} + +**External Documentation**: +- {Links to relevant library docs} + +**SQLSpec Guides**: +- [Architecture](../../docs/guides/architecture/architecture.md) +- [Testing](../../docs/guides/testing/testing.md) +- [Adapter Patterns](../../docs/guides/adapters/) diff --git a/specs/template-spec/recovery.md b/specs/template-spec/recovery.md new file mode 100644 index 000000000..360531b87 --- /dev/null +++ b/specs/template-spec/recovery.md @@ -0,0 +1,144 @@ +# Recovery Guide: {Feature Name} + +## To Resume Work + +1. **Read this document first** +2. Read [prd.md](prd.md) for full context +3. Check [tasks.md](tasks.md) for current progress +4. Review [research/](research/) for findings +5. Consult [AGENTS.md](../../AGENTS.md) for SQLSpec-specific patterns + +## Current Status + +**Phase**: {Current phase} +**Last Updated**: {Date} +**Completed**: {X/Y tasks} + +## Files Modified + +{List of files changed so far} + +## Next Steps + +{What should be done next} + +## Agent-Specific Instructions + +### For Expert Agent + +**Start Here**: +1. Read PRD (prd.md) +2. Review research questions at end of PRD +3. Consult AGENTS.md for SQLSpec standards: + - Type annotation standards (stringified, no __future__) + - Import standards (no nested imports) + - Clean code principles + - Adapter patterns + - driver_features patterns +4. Document findings in `research/` + +**Implementation Checklist**: +- [ ] Follow SQLSpec standards (see AGENTS.md) +- [ ] Stringified type hints for non-builtins +- [ ] No `from __future__ import annotations` +- [ ] Write tests as you go +- [ ] Test with affected adapters +- [ ] Update workspace continuously +- [ ] Will auto-invoke Testing agent when implementation complete +- [ ] Will auto-invoke Docs & Vision for docs and archival + +**SQLSpec-Specific Reminders**: +- Use protocols and type guards, not inheritance +- Follow parameter style abstraction patterns +- Test against real databases using pytest-databases +- Consider mypyc compilation impacts + +### For Testing Agent + +**Testing Strategy**: + +```bash +# Run tests +make test +# or +uv run pytest -n 2 --dist=loadgroup tests + +# Run with coverage +make coverage +# or +uv run pytest --cov -n 2 --dist=loadgroup + +# Run integration tests for specific adapter +uv run pytest tests/integration/test_adapters/test_asyncpg/ -v +``` + +**Test Patterns** (from AGENTS.md): + +- **MANDATORY**: Function-based tests (NOT class-based) +- Use descriptive test names +- Cover edge cases (empty, None, errors, boundaries) +- Test both success and failure paths +- Test all affected adapters +- Use pytest markers: `@pytest.mark.postgres`, `@pytest.mark.duckdb`, etc. + +### For Docs & Vision Agent + +**Documentation System**: + +- System: Sphinx +- Location: docs/ +- Build: `make docs` + +**Workflow**: + +1. **Documentation Phase**: + - Update API documentation (docs/reference/) + - Create/update guides in docs/guides/ + - Add working code examples + - Update CHANGELOG + +2. **Quality Gate Phase**: + - Verify all PRD acceptance criteria met + - Verify all tests passing + - Check AGENTS.md compliance + - BLOCK if criteria not met + +3. **Knowledge Capture Phase** (NEW!): + - Extract patterns from implementation + - Update AGENTS.md with new patterns + - Update relevant docs/guides/ with patterns + - Document with working examples + +4. **Re-validation Phase** (NEW!): + - Re-run tests after updates + - Rebuild docs (`make docs`) + - Check pattern consistency + - BLOCK if validation fails + +5. **Cleanup & Archive Phase**: + - Clean tmp/ directory + - Move to specs/archive/ + - Generate completion report + +## Blockers + +{Any blockers or dependencies} + +## Questions + +{Open questions for user or other agents} + +## SQLSpec-Specific Context + +**Adapters Affected**: +- {List adapters} + +**Core Components Affected**: +- {List components} + +**Mypyc Compatibility**: +- {Any special considerations} + +## Progress Log + +{Running log of changes - see tasks.md for details} diff --git a/specs/template-spec/research/.gitkeep b/specs/template-spec/research/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/specs/template-spec/tasks.md b/specs/template-spec/tasks.md new file mode 100644 index 000000000..ad7e75755 --- /dev/null +++ b/specs/template-spec/tasks.md @@ -0,0 +1,73 @@ +# Tasks: {Feature Name} + +## Phase 1: Planning & Research ✅ +- [x] 1.1 Create requirement workspace +- [x] 1.2 Write comprehensive PRD +- [x] 1.3 Create task breakdown +- [x] 1.4 Identify research questions + +## Phase 2: Expert Research +- [ ] 2.1 Research implementation patterns +- [ ] 2.2 Review adapter integration patterns +- [ ] 2.3 Analyze performance implications +- [ ] 2.4 Check mypyc compilation compatibility +- [ ] 2.5 Document findings in `research/` + +## Phase 3: Core Implementation (Expert) +- [ ] 3.1 Implement core functionality +- [ ] 3.2 Add adapter-specific logic +- [ ] 3.3 Update type definitions (stringified, no __future__) +- [ ] 3.4 Handle edge cases +- [ ] 3.5 Follow AGENTS.md standards + +## Phase 4: Integration (Expert) +- [ ] 4.1 Update affected adapters +- [ ] 4.2 Add necessary mixins +- [ ] 4.3 Update configuration +- [ ] 4.4 Test integrations + +## Phase 5: Testing (Testing Agent - Automatic) +- [ ] 5.1 Create unit tests (function-based) +- [ ] 5.2 Create integration tests (all adapters) +- [ ] 5.3 Test edge cases (empty, None, errors) +- [ ] 5.4 Performance validation +- [ ] 5.5 Coverage target achieved (>80%) + +## Phase 6: Documentation (Docs & Vision - Automatic) +- [ ] 6.1 Update API documentation +- [ ] 6.2 Create/update guide in docs/guides/ +- [ ] 6.3 Add code examples +- [ ] 6.4 Update CHANGELOG + +## Phase 7: Knowledge Capture & Archive (Docs & Vision - Automatic) +- [ ] 7.1 Extract new patterns from implementation +- [ ] 7.2 Update AGENTS.md with patterns +- [ ] 7.3 Update guides with new patterns +- [ ] 7.4 Re-validate (tests, docs, consistency) +- [ ] 7.5 Clean tmp/ and archive + +## Handoff Notes + +**To Expert**: +- Read PRD thoroughly +- Start with Phase 2 (Research) +- Consult AGENTS.md for SQLSpec-specific patterns +- Document findings before implementation +- Update tasks.md and recovery.md as you progress +- Will auto-invoke Testing and Docs & Vision agents when ready + +**To Testing Agent** (Auto-invoked by Expert): +- Follow SQLSpec test patterns (see AGENTS.md) +- Function-based tests ONLY (no class-based) +- Test all affected adapters +- Test edge cases (empty, None, errors) +- Achieve >80% coverage +- All tests must pass before returning control + +**To Docs & Vision** (Auto-invoked by Expert): +- Update project documentation +- Run quality gate (must pass) +- **Extract and capture new patterns in AGENTS.md** +- **Update docs/guides/ with new patterns** +- **Re-validate after updates** +- Clean and archive when complete