Skip to content

Parallel phase execution in builders #461

@waleedkadous

Description

@waleedkadous

Feature Request

Add support for parallel phase execution in builders. Currently SPIR/ASPIR builders execute phases strictly sequentially, but plans often contain dependency diagrams showing phases that could run concurrently.

Problem

A typical plan might have:

Phase 1: Data layer
Phase 2: API routes (depends on Phase 1)
Phase 3: UI components (depends on Phase 1)
Phase 4: Integration (depends on Phase 2, Phase 3)

Today: Phase 1 → 2 → 3 → 4 (serial, ~40 min)
With parallelism: Phase 1 → (2 + 3 parallel) → 4 (~30 min)

Proposed Design

  1. Plan format extension: Plans declare phase dependencies explicitly (not just ordering)
  2. Porch orchestration: Porch reads dependency graph and launches independent phases concurrently
  3. Multi-builder coordination: Each parallel phase gets its own builder in its own branch, merging results when all complete
  4. Conflict resolution: When parallel phases touch overlapping files, automated merge with consultation fallback
  5. Consultation gates: Phases that complete concurrently batch their consultations

Challenges

  • Git merge conflicts between parallel branches
  • Shared state — parallel phases may need to read each other's intermediate output
  • Consultation ordering — how to consult on phases that landed simultaneously
  • Progress tracking — dashboard needs to show parallel swimlanes
  • Failure handling — if one parallel phase fails, what happens to siblings?

This is a large project

Likely needs its own spec with multiple sub-phases. May warrant an EXPERIMENT protocol first to validate the multi-builder merge approach before full implementation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    projectNew project or featurespikeTechnical feasibility spike

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions