Typed Scala DSL for shell, expressions, steps, and step bundles

[russwyte]

Supersedes #46, which asked to move long run: strings into YAML resource files. That route was investigated and rejected (see the comment there): zio-blocks cannot round-trip a block scalar nested in a step sequence, and a steps file relocates the unchecked string rather than removing it, needing an interpolation language the moment it references a module id or an action pin.

Problem

Four layers of the API were stringly typed, and each one had already produced a real defect or a comment apologising for one:

1. Shell run: bodies. s-interpolated bash, so a literal $ in the script has to be written $$, and ZipxCentral carried a warning comment where a type belonged (a $$ that means "escaped dollar" to Scala and "PID" to bash). Planner.affectedScript had a .replace("\n\n", "\n") to clean up after stripMargin.
2. Expressions outside env: and if:. with: values, outputs:, and step env: all took raw ${{ … }} strings, so the planner hand-assembled contains(fromJson(needs.affected.outputs.modules), 'api') as text.
3. Step validity. Flat and all-optional (fixed by derives Schema and the on-disk mapping), so Step() and Step(run = …, uses = …) both compile and both render YAML GitHub rejects.
4. Reusable step groups. Bare StepContext => List[Step] lambdas: no name for a diagnostic, no way to concatenate two, no way to gate a group at once, no identity to publish.

What landed

A layer per problem, bottom-up, each independently useful:

• modules/shell (zipx.shell): a general shell AST. Script / Command / Word / ShTest over neotype-validated primitives. No zipx, GitHub, or zio-blocks dependency, so it is usable standalone. Command is an open trait with rendering as a method, deliberately not an enum: a consumer needing a construct zipx does not model implements it in their own build. The accepted cost is that a match over Command cannot be exhaustive.
• Expr in zipx-workflow: the GHA expression AST, with EnvValue and JobCondition delegating to it. Expr.Call over a FunctionName newtype, so an unknown function does not compile, which is what lets the planner's gates be built rather than interpolated.
• StepBuilder + render-time Step.validate: validity closed from both ends. Step.run(script) / Step.uses(ref) decide the mutually exclusive pair before any other field is set, and withInput exists only on the uses: builder, so with: on a run: step does not compile. Step.validate catches the other end (a hand-built or decoded step).
• Steps in zipx-core: a named, composable bundle that extends (StepContext => List[Step]), so every existing field accepts one with no signature change. ++ concatenates, when(JobCondition) gates the whole bundle, and zipx-core is on Central so an org bundle is an ordinary published value.

Validation is compile-time where it can be

Every DSL type is a neotype newtype, so an invalid value is unconstructible and an invalid literal fails during compilation with the validator's own message. Smart constructors are inline def so they forward a literal into the check, with make-style siblings returning Either[String, A] for genuinely runtime input.

The rules cover what the shell and GitHub actually do, not the convenient subset: no ' inside '…' (it cannot be escaped), no } inside ${…} (it closes the expansion early), no leading tab on a script line (YAML block-scalar indentation must be spaces), ExitCode 0-255 (the shell truncates modulo 256), secret names rejecting the reserved GITHUB_ prefix case-insensitively while still admitting GITHUB_TOKEN, output names rejecting the disabled set-output / save-state, uses: refusing an unpinned owner/repo.

sh"…" splices are Word*, so a bare String splice does not compile: string interpolation is exactly how the untyped hole would come back, and there is deliberately no implicit String => Word.

No thrown exceptions in the DSL

A failure is removed in the strongest way available at each site, in order: make it unrepresentable with a type (Block is a head plus a tail, so an empty if branch is unconstructible; InlineCommand is the subtype whose render is total, so a compound command in a pipeline leg does not compile); failing that, check a literal at compile time; failing that, return Either[String, A] naming the offending value. Throwing is confined to ZipxPlugin.orFail, the one place a zipx failure becomes an sbt build error, because sbt's task contract is to throw.

Escape hatches: allowed, typed, and loud

Script.raw / Expr.Raw / JobCondition.Raw stay, because a consumer who cannot express something must not be blocked. Raw holds List[ScriptLine], so the type guarantees raw content cannot emit YAML GitHub fails to parse (no \r, no C0 controls, no leading tab) and there is no separate lint pass to forget. What it can still produce is broken shell, so the text is reported via Command.rawFragments and zipxWorkflowGenerate logs a warning naming the bundle. A bare lambda reports nothing, which is the honest incentive to use Steps.built.

Acceptance: the generated YAML did not move a byte

Proven three ways, since a migration this wide is only credible if the output is identical:

1. Dogfood regenerate leaves git diff empty across ci.yml, zipx-action-pins-sync.yml and zipx-scala-steward.yml.
2. plugin/scripted zipx/generate-check passes. Independent, because its assertGraph asserts the literal gate strings (contains(fromJson(needs.affected.outputs.modules), 'api'), !cancelled(), startsWith(github.ref, 'refs/tags/v')) that Expr.Call now builds.
3. PlannerSpec / RenderSpec / the docs pages pass unmodified. An edit needed there would have meant output moved.

Zero-diff only proves nothing moved, not which script is which, so ScriptRenderSpec supplies the other half: each migrated script pinned against the exact string its pre-migration source produced. examples/monorepo was built against publishLocal to prove the consumer-facing types work outside this repo.

Docs

A Shell and steps page where every example is compiled and its rendered output asserted, including the examples that demonstrate a failure: that an invalid hand-built Step is Left, and that a raw fragment warns and names its bundle while a bare lambda does not. autoImport re-exports the DSL so a build.sbt needs no imports.

[russwyte]

Verified against the motivating consumer.

ConsumerStepsSpec (in zipx-core's tests) rebuilds chekhov's browser-setup steps in the DSL and asserts the result against chekhov's current run: bodies verbatim. Its steps were the useful kind of awkward: an if whose condition is a glob test on a command's exit status, $(…) substitutions nested inside double quotes with a three-stage pipeline inside one of them, sudo fronting another program, a ${HOME}-relative path assembled from parts, and a cache key mixing runner.os with hashFiles.

All of it expressed, byte-identically, so adopting the DSL there is a refactor and not a workflow diff to re-review. The spec also asserts what the lambda version could not promise: every step validates, the bundle reports no escape-hatch use, and one bundle assigned to both extraSteps and cacheRehydrateExtraSteps yields identical steps, which was #46's actual requirement.

Two findings from the exercise:

1. Two calls needed Word.vBraced ("${apt_mirror}") rather than Word.vq ("$apt_mirror") to match. The two spellings are only accidentally equivalent and chekhov wrote the explicit one, so the AST is right to distinguish them; noting it because it is the one place the DSL made an author's choice visible that a string had hidden.
2. ActionPins fields are String, so ctx.actions.cache cannot reach Step.uses's compile-time check and a build must call usesMake and handle an Either. Filed separately.