Pre-Flight Issue Feasibility Validator — Catch Doomed Pipelines Before They Start

[github-actions]

Strategic Improvement

Before launching any pipeline, run automated feasibility checks: git state is clean, issue description has actionable scope, dependencies are installable, test commands are valid, and no conflicting pipelines are running. This prevents wasted runs on fundamentally broken setups. Emit clear rejection reasons and suggest fixes.

Acceptance Criteria

• Validator checks git state, issue clarity score, dependency availability, test command validity
• Rejects pipelines that would obviously fail, with actionable error messages
• Logs rejection reasons to memory system for pattern analysis
• Integrates with daemon spawn logic (run before creating pipeline)
• Reduces failed pipeline starts by >30% within first week

Context

• Priority: P0
• Complexity: standard
• Generated by: Strategic Intelligence Agent
• Strategy alignment: P0: Reliability & Success Rate

[github-actions]

⚓ Shipwright Pipeline Starting

Field	Value
Template	standard
Complexity	standard
Max Iterations	20
Triage	Pre-flight feasibility checks directly improve pipeline reliability and cost efficiency by catching doomed runs early; clear integration point (daemon spawn), straightforward checks (git state, deps, test validity), minimal new external dependencies needed.
Runner	ubuntu-latest
Risk Level	unknown
Workflow	25601755138

The autonomous pipeline is processing this issue. Updates will be posted as stages complete.

[github-actions]

[github-actions]

[github-actions]

📋 Implementation Plan

Click to expand full plan (62 lines)

Perfect! I've created a comprehensive implementation plan for the Pre-Flight Issue Feasibility Validator. Here's the summary:

Plan Overview

Goal: Catch doomed pipelines before they start by running 5 feasibility checks on every GitHub issue before daemon spawns a pipeline.

Architecture

The validator runs before pipeline creation in the daemon, preventing wasted Claude tokens and context on obviously broken setups. It's exposed as both:

1. Daemon integration point — called from daemon_spawn_pipeline() before spawning
2. Standalone CLI — sw-pre-flight.sh --issue NUM --repo PATH for debugging/testing

5 Core Checks

1. Git State (30% weight) — Repo must be clean, no merge conflicts
2. Issue Clarity (20% weight) — Description heuristic scores well-formedness
3. Dependencies (25% weight) — npm ci --dry-run must succeed
4. Test Command (15% weight) — Test script must exist in package.json
5. Conflicting Pipelines (10% weight) — No other pipelines running this issue (warning-only)

Key Design Decisions

Decision	Why
Validation before pipeline creation	Avoid wasting tokens on doomed runs; ~30% expected reduction in failed pipeline starts
Exit codes: 0 (pass), 1 (reject), 2 (warnings)	Daemon can distinguish blocking failures from non-blocking issues
Async memory logging	Pre-flight doesn't block on memory write; logs rejection patterns for future analysis
Parallel check execution	All 5 checks run concurrently, total budget <3 seconds
Rule-based clarity scorer	Fast, deterministic, no API calls; tunable threshold to reduce false positives

Integration Points

• scripts/lib/daemon-dispatch.sh — Call pre-flight before daemon_spawn_pipeline()
• scripts/sw-memory.sh — Extend with memory_log_rejection() to track patterns
• scripts/sw-pre-flight.sh (NEW) — Main validator with 5 check functions
• scripts/sw-pre-flight-test.sh (NEW) — 19 tests covering all paths

20-Task Implementation Breakdown

Phase 1 (8 tasks): Core validator implementation
Phase 2 (3 tasks): Memory system integration
Phase 3 (3 tasks): Daemon integration
Phase 4 (3 tasks): Testing (unit, integration, E2E)
Phase 5 (3 tasks): Documentation & observability

Failure Mode Analysis (6 Critical Scenarios)

1. False positives (FM1) — Reject valid issues; mitigation: threshold tuning, CLI override, monitoring
2. Race condition on conflicts (FM2) — Collision after pre-flight passes; mitigation: atomic heartbeat, freshness check
3. Dependency inaccuracy (FM3) — npm passes pre-flight but fails build; mitigation: cache, retry, warnings
4. Test validation too lenient (FM4) — Broken test script not caught; mitigation: enhanced parsing, dry-run
5. Performance impact (FM5) — Pre-flight adds latency; mitigation: <3s timeout, parallelization, caching
6. Memory system failures (FM6) — Non-critical; mitigation: async logging, graceful degradation

Test Strategy

---

Generated by shipwright pipeline at 2026-05-09T13:09:08Z

[github-actions]

[github-actions]

[github-actions]

📐 Architecture Decision Record

Click to expand ADR (146 lines)

Design: Pre-Flight Issue Feasibility Validator — Catch Doomed Pipelines Before They Start

Context

Shipwright pipelines fail at ~35/week due to setup issues caught only after agent spawn: dirty git state, missing dependencies, invalid test commands, vague issues, and concurrent pipeline conflicts. Each failure burns 5–15k tokens and 10+ minutes before the root cause surfaces. The codebase already has hooks for this — sw-daemon.sh:1420 orchestrates spawn, sw-memory.sh:2240 records patterns, sw-pipeline.sh:279 is the manual entry point. We need a gating step before spawn_pipeline() that runs cheap, deterministic checks and rejects with actionable feedback.

Constraints: Bash 3.2 compatible (no associative arrays), set -euo pipefail, atomic file writes via tmp+mv, JSON via jq only, must respect NO_GITHUB/local mode, and must not block when daemon is in dry-run rollout.

Decision

A single orchestrator script scripts/sw-preflight.sh running 5 isolated check functions, invoked synchronously from sw-daemon.sh and sw-pipeline.sh before any agent spawn. Checks return uniform CheckResult JSON; the orchestrator aggregates by severity (CRITICAL→reject, WARNING→proceed+log) and emits a single preflight-result.json artifact plus a memory entry. Behavior is gated by two flags: SKIP_PREFLIGHT=true (bypass entirely) and PREFLIGHT_DRY_RUN=true (run + log, never reject).

Component Diagram

┌─────────────────────────────────────────────────────────────┐
│  Entry points                                               │
│  ├─ scripts/sw-daemon.sh   (autonomous queue dispatch)      │
│  └─ scripts/sw-pipeline.sh (manual `pipeline start`)        │
└──────────────────────┬──────────────────────────────────────┘
                       │ sw_preflight_validate()
                       ▼
┌─────────────────────────────────────────────────────────────┐
│  scripts/sw-preflight.sh (orchestrator)                     │
│  - Reads .claude/daemon-config.json for thresholds          │
│  - Runs checks sequentially, aggregates by severity          │
│  - Atomic write to .claude/pipeline-artifacts/              │
└────┬─────────┬──────────┬──────────┬──────────┬─────────────┘
     │         │          │          │          │
     ▼         ▼          ▼          ▼          ▼
  _check_   _check_   _check_    _check_    _check_
  git_      issue_    deps_      test_      conflict_
  state     clarity   installed  command    pipelines
                                            │
                       ┌────────────────────┘
                       ▼
┌─────────────────────────────────────────────────────────────┐
│  scripts/sw-memory.sh                                       │
│  └─ memory_log_preflight_rejection() (new)                  │
│     → ~/.shipwright/memory/<repo>/preflight-outcomes.jsonl  │
└─────────────────────────────────────────────────────────────┘

Dependencies point inward: orchestrator depends on checks; checks depend only on lib/compat.sh helpers. Daemon and CLI depend on orchestrator, never the reverse. No check function calls another check.

Interface Contracts

# Public — sw-preflight.sh
sw_preflight_validate(issue_number: str, goal_desc: str, repo_path: str)
  stdout: <JSON PreflightResult>
  exit:   0=pass | 1=reject | 3=fatal (repo missing, jq missing, etc.)
  errors: writes structured error to stderr; never partial JSON to stdout

# Private — each returns CheckResult JSON to stdout, exit 0 always
_check_git_state(repo_path) → {pass, severity, message, details, remediation}
_check_issue_clarity(desc, mode) → {pass, severity, score:0-100, missing[]}
_check_dependencies_installed(repo_path, timeout_ms) → {pass, severity, installer, error}
_check_test_command_valid(repo_path) → {pass, severity, command, dangerous_patterns[]}
_check_conflicting_pipelines(issue_number, repo_path) → {pass, severity, running_count}

</details>

---
_Generated by `shipwright pipeline` design stage at 2026-05-09T13:14:21Z_

[github-actions]

[github-actions]

🔨 Build started — shipwright loop with 20 max iterations, 1 agent(s), model: opus

[github-actions]

[github-actions]