RFC: Stage orchestration via pipeline-compose-run — output passing and failure policy

[aeswibon]

Context

I'm building pipeline-compose to run GitHub Actions workflows in order from a single pipeline definition — without requiring a compiled workflow file in the repo (unless you opt into compile).

v0.3.0 of pipeline-compose-run dispatches stages sequentially via workflow_dispatch, merges outputs from stage artifacts, and evaluates optional when: expressions before each stage.

Problem

For multi-stage releases (test → version sync → publish), native GitHub Actions pushes you into:

1. Large needs: graphs inside one workflow file — hard to debug when one job fails
2. Generated workflow YAML from compile tools — a second artifact to review and drift-track
3. workflow_run chains — awkward output passing; workflow renames break silently

Proposal

Treat orchestration as its own layer. Example from examples/run-tag-release:

name: pipeline
version: 1
stages:
  - id: ci
    workflow: .github/workflows/ci.yml

  - id: version-sync
    workflow: .github/workflows/stage-version-sync.yml
    needs:
      - ci
    outputs:
      - version

  - id: release-publish
    workflow: .github/workflows/stage-release-publish.yml
    needs:
      - version-sync
    inputs:
      version: ${{ context.version-sync.version }}

The run action:

• Dispatches each stage workflow via workflow_dispatch
• Waits for completion
• Collects outputs from artifact pipeline-compose-<stage-id>/outputs.json
• Resolves ${{ context.<stage-id>.<key> }} for downstream inputs and when: expressions

Compile (pipeline-compose-compile) remains optional for teams that need a static workflow with native needs: in review.

Schema: packages/core/schema/pipeline-v1.schema.json

Questions for feedback

1. Output schema — Is context.<stage-id>.<key> the right namespace, or would flatter aliases reduce friction?
2. Failure policy — Default fail-fast vs continue-on-error for independent stages?
3. Compile vs run — For your org, is committed generated YAML a hard requirement (compliance/review), or do you prefer runtime orchestration?
4. Artifact contract — Is pipeline-compose-<stage-id> + outputs.json predictable enough, or should stage workflows declare output keys in the pipeline file only?

Links

• Tag-release example: examples/run-tag-release
• Dev.to write-up: Why I Built pipeline-compose: Ordered GitHub Actions Without YAML Hell
• Tutorial: docs/tutorials/tag-release-pipeline.md

Substantive feedback on the output-passing model is especially useful before v0.4.0.

[aeswibon]

Update 2026-06-15: v1.0 GA + v1.1.0 shipped. Schema v2 only, pipeline-compose-export action, cross-repo stages, validate --simulate. See docs/migration/v1.0.md