-
Notifications
You must be signed in to change notification settings - Fork 1
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Context
This is Phase 3 of the create-expert improvement plan. Phase 1+2 (Expert Ecosystems + Usability Properties) is tracked in #337.
Problem
Currently, functional testing (happy/unhappy/adversarial) and usability testing run independently. We need:
- Coordinated test orchestration
- Trade-off analysis (functionality vs usability)
- Holistic quality assessment
Proposed Solution
Add Integration Manager
New expert that coordinates both functional and usability testing:
[experts."integration-manager"]
version = "1.0.0"
description = "Orchestrates coordinated functional and usability testing"
instruction = '''
You coordinate comprehensive testing across both functional and usability dimensions.
## Workflow
1. **Parallel Testing**: Run functional-manager and usability-manager concurrently
2. **Trade-off Analysis**: Identify conflicts (e.g., security vs ease-of-use)
3. **Integration Verification**: Ensure setup expert works with main expert
4. **Holistic Assessment**: Overall quality score across all dimensions
## Output
Integration test report with:
- Functional score (happy/unhappy/adversarial)
- Usability score (fresh-user/setup/demo/recovery)
- Trade-off analysis
- Recommendations for balance
'''
delegates = ["functional-manager", "usability-manager"]Consolidate Functional Managers
Merge happy-path-manager, unhappy-path-manager, adversarial-manager into one:
[experts."functional-manager"]
version = "1.0.0"
description = "Manages all functional PDCA cycles (happy/unhappy/adversarial)"
instruction = '''
You run comprehensive functional testing across all scenarios.
## PDCA Phases
### Phase 1: Happy Path
- Primary use cases
- Expected inputs
- Normal operation
### Phase 2: Unhappy Path
- Edge cases
- Invalid inputs
- Error handling
### Phase 3: Adversarial
- Prompt injection
- Path traversal
- Security attacks
Run all phases sequentially, report combined results.
'''
delegates = ["expert-tester"]Updated Workflow
create-expert
├── property-extractor (functional + usability properties)
├── ecosystem-builder (main + setup + demo + doctor)
├── integration-manager
│ ├── functional-manager (happy + unhappy + adversarial)
│ └── usability-manager (fresh-user + setup + demo + recovery)
└── report-generator (holistic quality report)
Success Criteria
- integration-manager coordinates parallel testing
- functional-manager consolidates 3 managers into 1
- Trade-off analysis identifies security vs usability conflicts
- Integration tests verify setup + main expert work together
- Report includes holistic quality score
Implementation Notes
- Integration-manager should run functional and usability in parallel
- Need conflict resolution strategy when functional and usability requirements clash
- Example trade-off: strict input validation (security) vs auto-correction (usability)
Dependencies
- Requires feat(create-expert): Generate Expert Ecosystems with Usability Focus #337 (Expert Ecosystems + Usability Manager) to be implemented first
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request