Replace sbt/setup-sbt in CI: Coursier or our own install step?

[pjfanning]

Our CI installs sbt with sbt/setup-sbt, and its signature verification step has now broken builds several times. I'd like to discuss replacing it, either with coursier/setup-action or with a small install step of our own.

What happened this time

On 2026-11-06, jobs in #3751 started failing in Install sbt, before sbt even started:

Error: compiling policy: unable to parse a policy, group or policySet from data

We pin sbt/setup-sbt to a commit SHA, but v1.5.13 fetches its verification policy from the default branch of carabiner-dev/policies. That policy added a meta.name field which, according to the policy repo's commit messages, ampel versions before v1.3.10 can't parse. setup-sbt uses ampel v1.3.6.

Three things made this worse:

• Pinning didn't protect us. The policy wasn't pinned, so a change in another repo broke our CI with no change on our side.
• It only shows up on a cache miss. Verification is skipped when the sbt download is cached, so builds kept passing for hours after the policy changed. The sbt-runner cache entry later disappeared, probably through eviction, and jobs started failing.
• Re-running doesn't help. The failing job doesn't save the cache, so nothing restores it. The latest Nightly Multi JVM Tests run on main failed with the same error. setup-sbt passes fail: false ("monitor only"), but that doesn't stop this failure.

setup-sbt v1.5.14 already fixes this by pinning the policy, and it was released before the breaking policy commit landed. We were still broken because Dependabot only checks weekly. So we need that update regardless of the outcome here. This discussion is about the longer term.

Earlier problems

All of these involve the same verification step:

• carabiner-dev/actions ampel/verify dependency broken in all recent releases (v1.5.2–v1.5.4) — ampel: command not found sbt/setup-sbt#118 (July 2026): ampel: command not found on a cache miss, affecting several releases. The cause was a bug in carabiner-dev/actions (ampel/verify (used by setup-sbt) creates literal $HOME/.carabiner/bin/ directory carabiner-dev/actions#97), and setup-sbt releases still pinned a version from before the fix.
• 1.5.10 pinned to broken carabiner action for self-hosted runner sbt/setup-sbt#142 (September 2026): v1.5.10 pinned a carabiner-dev/actions version that failed to load on a self-hosted runner.
• Pin signature policy to older format sbt/setup-sbt#155 (November 2026): the fix for this latest problem.

The maintainers have fixed these quickly, and verifying the sbt download is a good goal. The concern is the size of the dependency chain for downloading one zip file. An install step now pulls in carabiner-dev/actions, an ampel binary and a policy fetched at runtime. Any of those can stop every job in our CI, and other Pekko repos such as pekko-http, pekko-connectors, pekko-management and pekko-grpc use the same action.

Option 1: use coursier/setup-action

We already use this action in format.yml and link-validator.yml, it hasn't been a big problem for us, and it has no ampel or carabiner dependency. Its README shows installing sbt directly:

- uses: coursier/setup-action@<sha> # pin as usual
  with:
    apps: sbt

Benefits:

• Already familiar: it's in use here, and pekko-http, pekko-connectors, pekko-management and pekko-grpc each show a reference to it too.
• Simpler chain: it's a single Node action with no nested actions, so there's no ampel binary or runtime policy fetch.
• Dependabot still works: updates keep coming as normal action bumps, so there's no script to maintain.
• Easier on version branches: the ASF allow list has a '*' entry with keep: true for this action, which as I read it means older pins won't expire. sbt/setup-sbt versions get an expires_at date, which adds to the backport churn on branches like 1.x.

Costs and things to check:

• Still a third-party action: we'd still depend on someone else's release process.
• Verification is narrower: v3.1.0 adds an optional requireChecksum input, which checks the Coursier launcher against a SHA-256 pinned in the action, but only for the action's default Coursier version. We pin v3.0.4, which doesn't have it. It covers the launcher rather than the apps it installs.
• sbt version control: the app comes from the Coursier app channels, so we'd need to check how the sbt runner version is pinned and how that download is verified.

Proof of concept: apache/pekko-http#1401 makes this switch across that repo's workflows, using the v3.0.4 pin we already have. In its Checks job, the action ran cs install --contrib sbt, then sbt checkCodeStyle started sbt 1.13.1 and passed. Check headers and Code is formatted also passed. The Compile and test jobs were still running when I wrote this.

Option 2: our own install step with curl

Add a small local composite action, such as .github/actions/setup-sbt/action.yml, that downloads a fixed sbt runner version and checks it against a SHA-256 committed in our repo. sbt publishes .zip, .asc and .sha256 files with each release. The runner is only a launcher, and project/build.properties still selects the sbt version used for the build.

An untested sketch for Linux runners:

name: Install sbt
description: Install a pinned sbt runner and verify its checksum
runs:
  using: composite
  steps:
    - shell: bash
      env:
        SBT_VERSION: "<version>"
        SBT_SHA256: "<checksum committed here>"
      run: |
        zip="$RUNNER_TEMP/sbt-$SBT_VERSION.zip"
        curl -fsSL --retry 5 -o "$zip" \
          "https://github.com/sbt/sbt/releases/download/v$SBT_VERSION/sbt-$SBT_VERSION.zip"
        echo "$SBT_SHA256  $zip" | sha256sum --check --strict
        unzip -q "$zip" -d "$RUNNER_TEMP/sbt-$SBT_VERSION"
        echo "$RUNNER_TEMP/sbt-$SBT_VERSION/sbt/bin" >> "$GITHUB_PATH"

Benefits:

• Stronger failure behaviour: a mismatch always fails the job, and it runs on every build rather than only on a cache miss.
• Less to break: no runtime policy fetch and no extra actions or binaries.
• One place to update: we have 62 sbt/setup-sbt references across 39 workflow files on main, which could all point at one action.

Costs:

• We own it. Someone has to maintain the action, and Dependabot won't bump a version and checksum held in a script. We'd need a manual update or another way to automate it.
• Different trust model. A committed checksum pins exactly what we reviewed when we updated it. It doesn't check the sbt team's signature, so the update itself has to confirm the new file is genuine, for example by checking the .asc at that point.
• Download reliability. Without the action's cache, each job downloads from GitHub releases, so we may want to keep using actions/cache or accept the extra requests.
• Checkout order. A local action can only be used after actions/checkout, so any workflow that installs sbt first would need reordering.
• Sharing across repos. Other Pekko repos would either copy the action or reference it from apache/pekko by SHA.

Other alternatives

1. Keep sbt/setup-sbt and update faster. This is the least work, but it keeps the dependency chain, and a cold cache can still turn an upstream problem into a repo-wide failure.
2. Ask upstream for an option to turn verification off, or make it truly non-fatal. fail: false already suggests this is meant to be monitor only, so it could help everyone using the action. We could do this alongside any other option.

Questions

• Which is the better fit: Coursier, our own curl step, or staying with sbt/setup-sbt?
• How much verification of the sbt download do we need? For the curl step, is a committed checksum enough, or do we also want a GPG check of the .asc in CI?
• Should this cover just apache/pekko first, or be shared across the Pekko repos from the start?

[pjfanning]

Another recurring issue is in our version branches - 1.x, for instance.
I regularly have to update setup-sbt in backport PRs because the older versions stop working so quickly. It really is turning into a serious time sink.

[ptrdom]

In my projects I had good experience with coursier/setup-action (https://github.com/ptrdom/scalajs-esbuild/blob/main/.github/workflows/test.yml#L44-L50), though after I started pinning launcher versions I also moved project's sbt and launcher's sbt management over to Renovate to keep both on the same version - Renovate has custom regex resolution available, I understand that scala-steward cannot do that, Dependabot maybe. I also just noticed that I do not pin coursier/setup-action in my projects - should start doing that before it causes me trouble.

I would say:

1. Try coursier/setup-action, if the stability is not ideal - move to our own script then.
   1. For our own script:
      • Use actions/cache.
      • Figure out a way to automate version updates. If Dependabot cannot do it - that is a very good reason to migrate to Renovate. I bet there are ugly ways to hack together our own script to receive Dependabot updates with dummy/hook projects from which the workflow/script extracts versions, but maybe then it is easier to just migrate to Renovate.
2. I do not have a strong opinion on verification.
3. Implement it on one repo first and observe the stability for a while, if all good - unify the approach on all repos.