Skip to content

feat(create-expert): Add Integration Manager for coordinated functional+usability testing #339

@FL4TLiN3

Description

@FL4TLiN3

Context

This is Phase 3 of the create-expert improvement plan. Phase 1+2 (Expert Ecosystems + Usability Properties) is tracked in #337.

Problem

Currently, functional testing (happy/unhappy/adversarial) and usability testing run independently. We need:

  1. Coordinated test orchestration
  2. Trade-off analysis (functionality vs usability)
  3. Holistic quality assessment

Proposed Solution

Add Integration Manager

New expert that coordinates both functional and usability testing:

[experts."integration-manager"]
version = "1.0.0"
description = "Orchestrates coordinated functional and usability testing"
instruction = '''
You coordinate comprehensive testing across both functional and usability dimensions.

## Workflow

1. **Parallel Testing**: Run functional-manager and usability-manager concurrently
2. **Trade-off Analysis**: Identify conflicts (e.g., security vs ease-of-use)
3. **Integration Verification**: Ensure setup expert works with main expert
4. **Holistic Assessment**: Overall quality score across all dimensions

## Output

Integration test report with:
- Functional score (happy/unhappy/adversarial)
- Usability score (fresh-user/setup/demo/recovery)
- Trade-off analysis
- Recommendations for balance
'''
delegates = ["functional-manager", "usability-manager"]

Consolidate Functional Managers

Merge happy-path-manager, unhappy-path-manager, adversarial-manager into one:

[experts."functional-manager"]
version = "1.0.0"
description = "Manages all functional PDCA cycles (happy/unhappy/adversarial)"
instruction = '''
You run comprehensive functional testing across all scenarios.

## PDCA Phases

### Phase 1: Happy Path
- Primary use cases
- Expected inputs
- Normal operation

### Phase 2: Unhappy Path  
- Edge cases
- Invalid inputs
- Error handling

### Phase 3: Adversarial
- Prompt injection
- Path traversal
- Security attacks

Run all phases sequentially, report combined results.
'''
delegates = ["expert-tester"]

Updated Workflow

create-expert
├── property-extractor (functional + usability properties)
├── ecosystem-builder (main + setup + demo + doctor)
├── integration-manager
│   ├── functional-manager (happy + unhappy + adversarial)
│   └── usability-manager (fresh-user + setup + demo + recovery)
└── report-generator (holistic quality report)

Success Criteria

  • integration-manager coordinates parallel testing
  • functional-manager consolidates 3 managers into 1
  • Trade-off analysis identifies security vs usability conflicts
  • Integration tests verify setup + main expert work together
  • Report includes holistic quality score

Implementation Notes

  • Integration-manager should run functional and usability in parallel
  • Need conflict resolution strategy when functional and usability requirements clash
  • Example trade-off: strict input validation (security) vs auto-correction (usability)

Dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions