forked from sethdford/shipwright
-
Notifications
You must be signed in to change notification settings - Fork 0
Pipeline Plan 203
ezigus edited this page Mar 20, 2026
·
3 revisions
Plan stage complete. Here's the summary:
Approach: Source-based extraction (zero blast radius on callers)
3 new modules in scripts/lib/:
-
recruit-role-manager.sh(~900 lines): role definitions, matching, creation, evolution, self-tuning (12 functions) -
recruit-team-composer.sh(~500 lines): team assembly, routing, specializations, decomposition (4 functions) -
recruit-config-validator.sh(~900 lines): profiles, feedback, meta-learning, audit, stats (14 functions)
sw-recruit.sh shrinks to ~350 lines: shared state (paths, policy vars, utility functions) + sources the 3 modules + CLI router.
Key decisions:
- Modules are
sourced files, not standalone scripts — matches existinglib/pattern - Source order: role-manager → team-composer → config-validator (respects dependency chain)
- All 9 external callers (sw-pm.sh, sw-pipeline.sh, etc.) need zero changes — they invoke
sw-recruit.shas CLI - Section 18 test greps need updating to search
lib/recruit-*.shfiles
10 tasks, 3 new test files, full behavioral parity required before any new tests.
Full plan written to .claude/pipeline-artifacts/plan.md.
volution, self-tuning (~900 lines)
2. lib/recruit-team-composer.sh -> team assembly, routing, specializations, decomposition (~500 lines)
3. lib/recruit-config-validator.sh -> agent profiles, feedback, meta-learning, audit, stats (~900 lines)
4. sw-recruit.sh -> thin orchestrator: shared state + source modules + CLI router (~350 lines)
5. All 80+ existing tests pass with zero behavioral changes
6. New modular tests for each extracted component
Approach A: Source-based extraction (CHOSEN)
- Create
lib/recruit-role-manager.sh,lib/recruit-team-composer.sh,lib/recruit-config-validator.sh -
sw-recruit.shsources all three and keeps the CLI router - External callers unchanged -- zero blast radius on integrations
- Trade-offs: +simple, +safe, +incremental; -modules can still access each other's internals
Approach B: Standalone scripts with subprocess calls
- Each module is an independent script invoked via
bash - Trade-offs: +strong isolation; -breaks shared state, -requires serializing data between modules, -performance overhead, -major rewrite of internal data flow
Approach C: Single-file refactor with section markers
- Keep one file, add clear section comments
- Trade-offs: +zero risk; -doesn't achieve the goal of separate modules, -no independent testability
Decision: Approach A minimizes blast radius while achieving real modularity. Modules are sourced files (function libraries), not standalone scripts. This matches the existing lib/ pattern (e.g., lib/helpers.sh, lib/compat.sh, lib/pipeline-stages-build.sh).
| Risk | Impact | Mitigation |
|---|---|---|
| Function ordering dependency | Module B calls function from Module A before it's sourced | Source in strict order: shared, role-manager, team-composer, config-validator |
| Shared variable scope | Modules depend on ROLES_DB, PROFILES_DB, etc. | Keep all storage paths and policy vars in orchestrator, sourced first |
| Test breakage from path changes | Tests reference RECRUIT_SCRIPT directly | No change -- tests still invoke sw-recruit.sh which sources modules |
| Grep-based static tests break | Section 18 tests grep for functions in sw-recruit.sh | Update greps to also search lib/recruit-*.sh files |
| Missing function in wrong module | Subtle runtime error | Each module has a guard variable; integration test verifies all commands work |
What depends on sw-recruit.sh (9 callers -- all use CLI interface):
- sw-pm.sh -> team --json
- sw-pipeline.sh -> ingest-pipeline, match --json (via lib/pipeline-stages-build.sh)
- sw-triage.sh -> team --json
- sw-swarm.sh -> match --json
- sw-autonomous.sh -> match --json, team --json
- sw-loop.sh -> team --json
None of these need changes -- they all invoke sw-recruit.sh as a CLI, and the CLI router stays in that file.
Internal dependencies between the three modules:
- recruit-team-composer.sh calls: initialize_builtin_roles(), _recruit_keyword_match(), _recruit_has_claude(), _recruit_call_claude() -- from role-manager + shared
- recruit-config-validator.sh calls: ensure_recruit_dir(), _recruit_locked_write(), initialize_builtin_roles(), _recruit_track_role_usage() -- from shared + role-manager
- No circular dependencies exist
+------------------------------------------------------------------+
| sw-recruit.sh (orchestrator) |
| Shared state (paths, policy vars, utility functions) |
| CLI router (case statement) |
| Sources: lib/compat.sh, lib/helpers.sh, sw-intelligence.sh |
| Sources: lib/recruit-role-manager.sh |
| Sources: lib/recruit-team-composer.sh |
| Sources: lib/recruit-config-validator.sh |
+--------+-----------------+-----------------+---------------------+
| source | source | source
v v v
+------------------+ +------------------+ +----------------------+
| recruit-role- | | recruit-team- | | recruit-config- |
| manager.sh | | composer.sh | | validator.sh |
| | | | | |
| Role defs | | cmd_team | | cmd_record_outcome |
| Matching | | cmd_route | | cmd_ingest_pipeline |
| cmd_roles | | cmd_specialize | | cmd_evaluate |
| cmd_match | | cmd_decompose | | cmd_profiles |
| cmd_create_role | | | | cmd_promote |
| cmd_evolve | | Depends on: | | cmd_onboard |
| cmd_invent | | role-manager | | cmd_mind |
| cmd_self_tune | | (matching fns) | | cmd_reflect |
| | | | | cmd_audit |
| No deps on | | | | cmd_stats |
| other modules | | | | cmd_help |
+------------------+ +------------------+ | |
| Depends on: |
| role-manager |
| (role usage fns) |
+----------------------+
External callers (unchanged):
sw-pm.sh, sw-pipeline.sh, sw-triage.sh, sw-swarm.sh,
sw-autonomous.sh, sw-loop.sh
All invoke: bash sw-recruit.sh <cmd> [args]
// lib/recruit-role-manager.sh -- Role Assignment, Matching, Evolution
// Depends on: shared state (ROLES_DB, HEURISTICS_DB, MATCH_HISTORY, etc.)
initialize_builtin_roles(): void
cmd_roles(): void
cmd_create_role(args: string[]): void
_recruit_keyword_match(task: string): string
_recruit_llm_match(task: string, roles_json: string): string
_recruit_record_match(task, role, method, conf, agent_id): string
cmd_match(args: string[]): void
cmd_evolve(): void
_recruit_track_role_usage(role: string, outcome: string): void
cmd_invent(): void
cmd_self_tune(): void
_recruit_compute_population_stats(): void
// lib/recruit-team-composer.sh -- Team Assembly, Routing, Decomposition
// Depends on: role-manager (initialize_builtin_roles, _recruit_keyword_match)
cmd_team(args: string[]): void
cmd_route(args: string[]): void
cmd_specializations(): void
cmd_decompose(args: string[]): void
// lib/recruit-config-validator.sh -- Profiles, Feedback, Meta-Learning, Audit
// Depends on: role-manager (_recruit_track_role_usage, initialize_builtin_roles)
cmd_record_outcome(args: string[]): void
cmd_ingest_pipeline(args: string[]): void
cmd_evaluate(args: string[]): void
cmd_profiles(): void
cmd_promote(args: string[]): void
cmd_onboard(args: string[]): void
cmd_mind(args: string[]): void
_recruit_meta_learning_check(agent_id, outcome): void
cmd_reflect(): void
_recruit_reflect(): void
_recruit_meta_validate_self_tune(accuracy): void
cmd_audit(): void
cmd_stats(): void
cmd_help(): voidUser/Script invokes: bash sw-recruit.sh <command> [args]
|
+------+------+
| Orchestrator |
| (sw-recruit) |
| 1. Load env |
| 2. Source |
| modules |
| 3. Route cmd |
+------+------+
+-------------+-------------+
v v v
role-manager team-composer config-validator
| | |
v v v
~/.shipwright/recruitment/ (shared JSON data stores)
- Each cmd_* function handles its own argument validation and prints usage errors
- LLM failures gracefully degrade to keyword/heuristic fallback (within role-manager and team-composer)
- File I/O errors caught by set -euo pipefail + ERR trap (stays in orchestrator)
- The orchestrator case statement handles unknown commands
- No errors cross module boundaries -- each function is self-contained
- Lines 1-150: shebang, pipefail, SCRIPT_DIR, version, dependency check, compat/helpers source, fallbacks, _recruit_locked_write, storage paths, policy integration, policy vars, ensure_recruit_dir, intelligence source, _recruit_has_claude, _recruit_call_claude
- Lines 2607-2645: Main CLI router
- NEW: Three source statements for the extracted modules
- initialize_builtin_roles (156-290)
- _recruit_keyword_match (297-354)
- _recruit_llm_match (357-386)
- _recruit_record_match (390-431)
- cmd_create_role (437-553)
- _recruit_track_role_usage (746-763)
- cmd_evolve (766-876)
- _recruit_compute_population_stats (882-902)
- cmd_self_tune (1755-1872)
- cmd_roles (1878-1888)
- cmd_match (1890-1999)
- cmd_invent (1376-1488)
- cmd_specializations (909-952)
- cmd_route (955-1000)
- cmd_team (1006-1181)
- cmd_decompose (1654-1749)
- cmd_record_outcome (560-680)
- cmd_ingest_pipeline (683-740)
- _recruit_meta_learning_check (1187-1236)
- cmd_reflect (1239-1246)
- _recruit_reflect (1248-1321)
- _recruit_meta_validate_self_tune (1325-1370)
- cmd_mind (1494-1648)
- cmd_evaluate (2002-2069)
- cmd_profiles (2071-2085)
- cmd_promote (2087-2163)
- cmd_onboard (2165-2259)
- cmd_stats (2261-2316)
- cmd_help (2318-2369)
- cmd_audit (2383-2605)
- Existing tests (80+ in sw-recruit-test.sh): All pass unchanged -- behavioral parity proof
- New unit tests (~30): 10 per module, testing functions in isolation
- sw-role-manager-test.sh: role init, keyword matching, match recording, role creation, self-tune
- sw-team-composer-test.sh: team heuristic composition, JSON output, route logic, decompose
- sw-config-validator-test.sh: record-outcome, profile updates, ingest-pipeline, audit scoring
- Integration tests (~5): Orchestrator sourcing all modules, cross-module flows
All cmd_* functions tested via existing tests (behavioral parity). New tests focus on recruit* internal functions.
- Happy path: match --json returns valid JSON with primary_role, model, confidence
- Error case: missing arguments produces usage error
- Error case: LLM unavailable triggers keyword fallback with correct role
- Edge case: empty roles DB triggers initialize_builtin_roles
- Edge case: concurrent writes handled by _recruit_locked_write
| Action | File |
|---|---|
| CREATE | scripts/lib/recruit-role-manager.sh |
| CREATE | scripts/lib/recruit-team-composer.sh |
| CREATE | scripts/lib/recruit-config-validator.sh |
| MODIFY | scripts/sw-recruit.sh |
| CREATE | scripts/sw-role-manager-test.sh |
| CREATE | scripts/sw-team-composer-test.sh |
| CREATE | scripts/sw-config-validator-test.sh |
| MODIFY | scripts/sw-recruit-test.sh |
- Create lib/recruit-role-manager.sh: Extract role management functions with source guard (_RECRUIT_ROLE_MANAGER_LOADED)
- Create lib/recruit-team-composer.sh: Extract team composition functions with source guard (_RECRUIT_TEAM_COMPOSER_LOADED)
- Create lib/recruit-config-validator.sh: Extract validation/feedback functions with source guard (_RECRUIT_CONFIG_VALIDATOR_LOADED)
- Refactor sw-recruit.sh: Remove extracted functions, add three source statements after shared state initialization
- Update sw-recruit-test.sh: Fix Section 18 static grep tests to also search lib/recruit-*.sh files
- Run existing tests: Verify all 80+ tests pass (behavioral parity proof)
- Create sw-role-manager-test.sh: Unit tests for role-manager functions (source the module directly)
- Create sw-team-composer-test.sh: Unit tests for team-composer functions
- Create sw-config-validator-test.sh: Unit tests for config-validator functions
- Run full test suite: npm test to verify no regressions
- Task 1: Create scripts/lib/recruit-role-manager.sh with extracted role management functions
- Task 2: Create scripts/lib/recruit-team-composer.sh with extracted team composition functions
- Task 3: Create scripts/lib/recruit-config-validator.sh with extracted validation/feedback functions
- Task 4: Refactor scripts/sw-recruit.sh into thin orchestrator
- Task 5: Update scripts/sw-recruit-test.sh Section 18 static greps
- Task 6: Run existing sw-recruit-test.sh -- verify all 80+ tests pass
- Task 7: Create scripts/sw-role-manager-test.sh with ~10 unit tests
- Task 8: Create scripts/sw-team-composer-test.sh with ~10 unit tests
- Task 9: Create scripts/sw-config-validator-test.sh with ~10 unit tests
- Task 10: Run npm test to verify full suite passes
- Tasks 1-3: Independent, can run in parallel
- Task 4: Blocks on Tasks 1-3
- Task 5: Blocks on Task 4
- Task 6: Blocks on Tasks 4-5
- Tasks 7-9: Block on Task 6 (verify parity before writing new tests)
- Task 10: Blocks on Tasks 7-9
| Risk | What could break | Mitigation |
|---|---|---|
| Source order wrong | Functions undefined at call time | Source role-manager, team-composer, config-validator (in that order) |
| Shared variable not visible | Module cant access ROLES_DB etc. | All shared state in orchestrator before sourcing |
| Section 18 greps fail | Tests grep for functions in sw-recruit.sh only | Update greps to also search lib/recruit-*.sh |
| Double-source | Functions redefined | Guard variable in each module |
| set -euo pipefail in modules | Conflicting error handling | Dont add separate pipefail -- modules inherit from orchestrator |
| Approach | Complexity | Blast Radius | Achieves Goal |
|---|---|---|---|
| A: Source-based (chosen) | Low | Zero on callers | Yes |
| B: Subprocess scripts | High | Medium | Yes (stronger isolation) |
| C: Section comments only | None | Zero | No |
- Three new lib/recruit-*.sh files exist with extracted functions
- sw-recruit.sh is < 400 lines (down from 2645)
- All 80+ existing tests in sw-recruit-test.sh pass unchanged
- New test files exist: sw-role-manager-test.sh, sw-team-composer-test.sh, sw-config-validator-test.sh
- npm test passes with no regressions
- No external caller changes required (sw-pm.sh, sw-pipeline.sh, etc.)
- All modules have source guards to prevent double-loading
- Bash 3.2 compatible (no associative arrays, no readarray, etc.)