Skip to content

End-to-end validation of v2 extract/install across multiple AI tools #82

@troylar

Description

@troylar

Description

Validate the v2 extract → install pipeline end-to-end with real projects and real AI tools. Create integration tests and a manual test matrix that exercises extraction from real-world project configs and installation into multiple tool formats.

Context

v2 (0.12.0) shipped with AI-powered extraction and installation, but testing has been primarily unit-level with mocked LLM responses. We need to verify:

  1. Extraction produces valid practice declarations from real project configs
  2. Installation correctly translates practices to each tool's native format
  3. AI adaptation produces sensible merges with existing rules
  4. File-copy fallback (--no-ai) works correctly for all tools
  5. Round-trip fidelity: extract from tool A, install to tool B, content is meaningful

Acceptance Criteria

  • Integration test suite that runs extract → install for at least 3 tool pairs (e.g., Claude→Cursor, Cursor→Kiro, Claude→Copilot)
  • Test with real project configs (use devsync's own .claude/rules/ as source)
  • Verify file placement, extensions, and content format for each target tool
  • Test --no-ai mode produces valid output for all tools
  • Test v1 backward compatibility (ai-config-kit-package.yaml packages still install)
  • Document a manual test matrix for pre-release validation
  • Tests pass: invoke test-unit && invoke test-integration

Test Matrix

Source Target Mode Status
Claude Code Cursor AI
Claude Code Kiro AI
Claude Code Copilot AI
Claude Code Cursor --no-ai
Cursor Claude Code AI
v1 package Claude Code file-copy
v1 package Cursor file-copy

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions