Skip to content

Pipeline Design 37

Jump to bottom
ezigus edited this page Mar 1, 2026 · 1 revision

Design: chore: sync fork with upstream sethdford/shipwright (v3.1.0 → v3.2.0, 27 commits)

Context

The fork has diverged from upstream sethdford/shipwright by 38 local commits while upstream has advanced 42 commits from the merge base to v3.2.4 (not v3.2.0 as originally stated). Critical local changes must be preserved:

  1. Local fix #25: Removed hardcoded npm test from pipeline templates (autonomous, cost-aware, full)
  2. Local fix #22: Disabled per-iteration test_cmd passing in sw-loop.sh (stage_intake cleanup)
  3. Local fix #23: Added multi-language detection to sw-pipeline-detection.sh
  4. Multiple shellcheck improvements from upstream (new SC2312 directives, set -euo pipefail patterns)
  5. Upstream additions: New audit stage in templates, full model names (claude-opus-4-6, claude-sonnet-4-6)

The merge has 11 conflicting files (22 auto-merge cleanly). Conflict resolution strategy is deterministic:

  • Keep local (3): .claude/CLAUDE.md (thin wrapper only), 2 ephemeral caches
  • Merge both (8): Accept upstream's architectural changes while preserving local functional fixes

Decision

Perform a strategic 3-way merge with explicit conflict resolution per file, rather than simple rebase or fast-forward:

  1. Merge with conflict markers into a feature branch (ci/sync-upstream-v3.2.4)
  2. Resolve conflicts systematically using the conflict matrix below
  3. Verify version consistency (package.json, README, scripts/VERSION all at 3.2.4)
  4. Run full test suite before PR (all 102 test suites, not just smoke tests)
  5. Manual validation of critical paths (pipeline startup, daemon spawn, model routing)

Conflict Resolution Matrix

File Strategy Rationale
.claude/CLAUDE.md Keep local (thin wrapper) Upstream only updated auto-generated line count comments; local wrapper is canonical
.claude/pipeline-artifacts/* Delete both (ephemeral) Rebuild on first run; never commit artifacts
scripts/sw-pipeline-detection.sh Merge: upstream shellcheck + local multi-lang detection Upstream: SC2312 fix; local: [[ $lang =~ (ts|js|go) ]] pattern for detection
scripts/sw-loop.sh Merge: upstream error handling + local stage_intake.sh removal Upstream: new error-summary.json; local: removed pass_test_cmd logic
scripts/sw-otel.sh Merge: upstream OTEL enrichment + local perf pattern Upstream: new OpenTelemetry spans; local: emit_event batching
scripts/sw-pipeline.sh Merge: upstream compose + local goal compaction Upstream: new compose logic; local: goal="${goal:0:140}" truncation
scripts/sw-hygiene.sh Merge: upstream rule enforcement + local cleanup logic Upstream: new architecture check; local: orphaned .claude/ cleanup
scripts/sw-tmux-status.sh Merge: upstream pane-border-status + local role color Upstream: new status bar format; local: set-option -p @pane-role
templates/pipelines/autonomous.json Merge: remove hardcoded test_cmd, add audit stage, use full model names Keep local fix #25 (test_cmd removed), accept upstream's new audit stage and claude-opus-4-6
templates/pipelines/cost-aware.json Merge: same strategy Same rationale
templates/pipelines/full.json Merge: same strategy Same rationale

Alternatives Considered

  1. Rebase local commits onto upstream — Pros: cleaner history; Cons: re-runs CI for each local commit, loses context from original merge base, risky if any commit touches templates
  2. Cherry-pick only critical upstream commits — Pros: minimal conflict surface; Cons: misses audit stage, model routing improvements, and upstream bug fixes; breaks traceability
  3. Squash local commits, then merge — Pros: one PR; Cons: loses 38 commits' worth of reasoning, makes blame tracking harder, doesn't preserve fix attribution
  4. Keep fork diverged indefinitely — Pros: no merge risk; Cons: accumulating technical debt, miss upstream security fixes, duplicate effort on shellcheck cleanup

Chosen: Option 1 (3-way merge with conflict resolution) balances safety (explicit conflict handling), completeness (gets all upstream fixes), and auditability (preserves both sides' history).

Implementation Plan

Phase 1: Preparation (Pre-Merge)

  • Save current HEAD to a checkpoint: shipwright checkpoint save "pre-3.2.4-merge"
  • Verify local test suite passes (all 102 tests must pass before merge)
  • Confirm git status is clean (no uncommitted changes)
  • Fetch upstream: git fetch upstream (or equivalent remote)

Phase 2: Execute Merge

  • Create feature branch: git checkout -b ci/sync-upstream-v3.2.4
  • Dry-run merge with conflict report: git merge upstream/main --no-commit --no-ff (or upstream's main branch)
  • Identify all 11 conflicted files in git status

Phase 3: Resolve Conflicts (Per Table Above)

  • .claude/CLAUDE.md: Keep local (ours) — git checkout --ours
  • .claude/pipeline-artifacts/*: Delete both — git rm (ephemeral)
  • scripts/sw-pipeline-detection.sh: Manual merge (preserve local [[ $lang =~ ... ]], add upstream # SC2312: ...)
  • scripts/sw-loop.sh: Manual merge (keep local stage_intake removal, add upstream error-summary.json)
  • scripts/sw-otel.sh: Manual merge (local event batching + upstream span enrichment)
  • scripts/sw-pipeline.sh: Manual merge (local goal compaction + upstream compose)
  • scripts/sw-hygiene.sh: Manual merge (local cleanup + upstream rule enforcement)
  • scripts/sw-tmux-status.sh: Manual merge (local role color + upstream status format)
  • templates/pipelines/autonomous.json: Remove "test_cmd": "npm test", add "audit" stage, set "model": "claude-opus-4-6"
  • templates/pipelines/cost-aware.json: Same
  • templates/pipelines/full.json: Same

Phase 4: Post-Merge Validation

  • Verify version consistency: scripts/check-version-consistency.sh or shipwright version check (must read 3.2.4 everywhere)
  • Stage the merge commit: git add -A && git commit -m "chore: merge upstream v3.2.4, resolving 11 conflicts"
  • Full test suite: npm test (all 102 test suites must pass)
  • Smoke test pipeline startup: shipwright status --json (should show clean state)
  • Smoke test daemon health: shipwright daemon start --local --max-iterations 1 (should spawn, complete, exit cleanly)
  • Validate model routing: Check scripts/sw-model-router.sh loads claude-opus-4-6, claude-sonnet-4-6 correctly

Phase 5: Open PR

  • Push to origin: git push -u origin ci/sync-upstream-v3.2.4
  • Create PR: Title = "chore: sync fork with upstream v3.2.4 (42 commits, 11 conflicts resolved)"
  • Link to issue #37
  • Assign reviewer (likely maintainer familiar with recent changes)
  • Merge PR with squash (to keep main branch history clean) or regular merge (to preserve fork history)

Files to Create

  • None (this is pure merge, no new files)

Files to Modify

  • Conflicts (11 files above)
  • No new dependencies — upstream doesn't introduce new npm packages; all additions are shell scripts

Risk Areas

  1. Manual merge conflicts in shell scripts — Risk: typos in function logic if manually merging. Mitigation: After resolving each script, run bash -n scripts/sw-*.sh (syntax check) before committing.
  2. Template JSON syntax — Risk: trailing comma, missing bracket after edit. Mitigation: jq . templates/pipelines/*.json > /dev/null before commit.
  3. Version inconsistency after merge — Risk: one file stays at 3.1.0, causing CI failure. Mitigation: shipwright version check must pass; if it fails, manually grep for old version strings.
  4. Upstream CI not yet passing — Risk: we merge upstream, upstream's own tests fail upstream. Mitigation: Check upstream repo's latest CI status before merge; if red, coordinate with upstream maintainer.
  5. Local fixes lost in merge — Risk: if we accidentally take upstream version of a file with a local fix. Mitigation: Verify each resolved file against the conflict matrix; double-check fix #25 (test_cmd removal) is in all 3 templates.

Validation Criteria

  • Git merge completes without unresolved conflicts (all markers removed, git status shows staged changes only)
  • Version consistency — shipwright version check exits 0 (3.2.4 in package.json, README, all scripts)
  • Full test suite passes — npm test shows 0 failures across all 102 test suites
  • Pipeline CLI works — shipwright pipeline start --issue 999 --local can execute intake + plan stages without error
  • Daemon can spawn — shipwright daemon start --local --max-parallel 1 spawns at least one worker, processes one issue
  • Model routing reflects upstream — scripts/sw-model-router.sh reads claude-opus-4-6, claude-sonnet-4-6 from updated daemon config
  • Fix #25 preserved — Inspect templates/pipelines/autonomous.json (and cost-aware, full) — "test_cmd" key is absent (not present in JSON)
  • Audit stage present — All 3 templates include "audit" in stage sequence
  • Shellcheck directives merged — scripts/sw-pipeline-detection.sh includes upstream # SC2312: use -x with printf and local multi-lang pattern
  • PR CI passes — GitHub Actions workflows (if any) pass on the feature branch before merge

Architecture Rationale: This is a surgical 3-way merge, not a rebase or cherry-pick. It preserves 38 local commits' worth of fixes while cleanly absorbing 42 upstream commits' improvements. The conflict matrix ensures deterministic, auditable resolution — each decision is documented and testable. The 5-phase plan reduces risk by validating at each step before proceeding.

Clone this wiki locally