Skip to content

Add container build backend and build verify command#2525

Draft
leighmcculloch wants to merge 66 commits intomainfrom
feat/reproducible-builds-via-docker
Draft

Add container build backend and build verify command#2525
leighmcculloch wants to merge 66 commits intomainfrom
feat/reproducible-builds-via-docker

Conversation

@leighmcculloch
Copy link
Copy Markdown
Member

@leighmcculloch leighmcculloch commented Apr 27, 2026

What

Add --backend docker[=<image>] to stellar contract build (and deploy/upload) that runs the entire build pipeline inside a container whose entrypoint is stellar. Add a stellar contract build verify subcommand that reads everything it needs from the wasm's metadata, rebuilds, and reports which (if any) rebuilt artifact is byte-identical to the original. Add a mainnet warning on stellar contract deploy when the wasm is missing the meta entries needed for independent verification.

Why

Contract builds vary across host OS, architecture, and toolchain, preventing third parties from independently confirming a deployed contract was built from given source. Pinning the build to a docker image plus the rust toolchain version makes builds reproducible, recording the source repo + commit + per-package build options lets verifiers rebuild the exact same artifact, and the new verify subcommand automates the rebuild-and-compare check.

Closes #2506.

How it works

Three parts: build-time recording, deploy-time warning, and verify-time reproduction.

Build

stellar contract build --backend local                          # default; host build
stellar contract build --backend docker                         # build inside docker.io/stellar/stellar-cli@sha256:...
stellar contract build --backend docker=stellar/stellar-cli:26.0.0
stellar contract build --backend docker=quay.io/myorg/myimage@sha256:...

For all backends (including local), the build:

  • Detects whether the workspace is a clean git checkout. If clean and there's an origin remote, embeds source_repo (URL canonicalized to https://…), source_rev (full HEAD SHA), and per-package build options (bldopt_manifest_path relative to git root, bldopt_package, bldopt_profile, optional bldopt_optimize). The manifest path is auto-inserted whether or not --manifest-path was passed on the CLI.
  • If the working tree has uncommitted changes, prints a warning and omits source_repo / source_rev / bldopt_*.
  • If not a git repo, silently omits.

For --backend docker, additionally:

  1. Resolves the requested image. Default is docker.io/stellar/stellar-cli@sha256:cb2fc3116a6ace37a77ca6bb88afb4bee57fc746cd556a4373f2c3ee95d4e917 — pinned by digest so the recorded bldimg is reproducible from day one and we sidestep the longstanding Apple Silicon docker quirk where pulling a multi-arch tag with --platform=linux/amd64 leaves RepoDigests empty after pull.
  2. Pulls the image (skipping the pull if it's already locally present, since digest-pinned references are immutable).
  3. Bind-mounts on the container:
    • <git_root or workspace_root>/source (rw, source — also where cargo writes its target dir, shared with the host)
    • host ~/.cargo/registry/usr/local/cargo/registry (rw, cached crate downloads)
  4. The container runs as the host uid:gid, so files written to the bind mount are readable/writable by the host user.
  5. Overrides the image's entrypoint to invoke stellar directly, bypassing the official image's entrypoint.sh (which launches dbus + gnome-keyring and trips when running under a host UID with no /etc/passwd entry — see Docker image's entrypoint dbus init fails when run as non-root UID #2543). contract build doesn't use the keyring, so the wrapper is irrelevant here.
  6. Runs stellar contract build --manifest-path /source/<rel> --profile <p> --locked --meta bldimg=<digest> [forwarded args] inside the container. The args use only flags that exist in published stellar/stellar-cli images today; no new flags are added, and --backend local is deliberately not passed (it's a flag added in this PR and isn't recognized by published images).
  7. The in-container cli does cargo + meta injection + spec filtering + optional wasm-opt itself; the host only orchestrates and copies outputs to --out-dir if requested.

The wasm's contractmetav0 custom section is populated with up to nine entries:

key value regex (validation) injected by
cliver 26.0.0#abc1234… (CLI version + git rev) ^\d+\.\d+\.\d+(-[A-Za-z0-9.+-]+)?#([0-9a-f]{40}(-dirty)?)?$ stellar-cli
bldimg docker.io/stellar/stellar-cli@sha256:… ^[^@\s]+@sha256:[0-9a-f]{64}$ stellar-cli (this PR; only with --backend docker)
rsver 1.83.0 (resolved rustc version) ^\d+\.\d+\.\d+(-[A-Za-z0-9.+-]+)?$ soroban-sdk
source_repo https://github.com/user/repo (clean repo's origin) ^https?://\S+$ stellar-cli (this PR)
source_rev full 40-char HEAD SHA ^[0-9a-f]{40}$ stellar-cli (this PR)
bldopt_manifest_path e.g. contracts/foo/Cargo.toml (relative to git) ^([^/\s]+/)*Cargo\.toml$ stellar-cli (this PR)
bldopt_package cargo package name being built ^[A-Za-z][A-Za-z0-9_-]*$ stellar-cli (this PR)
bldopt_profile cargo profile (e.g. release) ^[A-Za-z][A-Za-z0-9_-]*$ stellar-cli (this PR)
bldopt_optimize true (only present when --optimize was used) ^true$ stellar-cli (this PR)

The presence of bldimg is what distinguishes a docker build from a local one — there's no separate bldbkd field. For full reproducibility from day one, pin to a specific image with --backend docker=<name>@sha256:… and commit before building.

--backend and --docker-host are also exposed on stellar contract deploy and stellar contract upload (which auto-build when no --wasm / --wasm-hash is given), so the same flags work end-to-end.

Deploy

stellar contract deploy against mainnet now warns when the wasm is missing any of cliver, bldimg, rsver, source_repo, source_rev, bldopt_manifest_path, bldopt_package, bldopt_profile:

⚠ the wasm being deployed is missing reproducibility meta entries: ["bldimg", "source_repo", "source_rev", "bldopt_manifest_path", "bldopt_package", "bldopt_profile"]. The deployed wasm may not be independently verifiable. To make it reproducible, build with `stellar contract build --backend docker` in a clean git repository.

The check is mainnet-only (matches network passphrase against Public Global Stellar Network ; September 2015); on testnet/futurenet/local the wasm deploys silently.

Verify

verify is a subcommand of build — it lives at stellar contract build verify, and works on multi-contract workspaces by rebuilding and finding the match.

stellar contract build verify --contract-id CXXX… --network mainnet
stellar contract build verify --wasm-hash <hash>  --network mainnet
stellar contract build verify --wasm contract.wasm
  1. Fetches the original wasm (file path, hash, or contract id, same flags as contract info).
  2. Reads cliver, bldimg (optional), rsver, and bldopt_* (optional, best-effort) from the wasm's meta. Missing bldopt_* entries trigger a warning rather than an error and the build falls back to its defaults — verify still runs, just with the caveat that the rebuild may not be reproducible.
  3. Picks the rebuild backend from the meta:
    • bldimg present → Backend::Docker { image: bldimg }. The image's pinned digest pulls the same in-container cli that produced the original.
    • bldimg absent → Backend::Local. Best-effort rebuild on the host.
  4. Forwards the wasm's rsver to the rebuild as RUSTUP_TOOLCHAIN (in-container) or cargo +<rsver> (local). For docker the toolchain inside the image is fixed by whoever built it; passing RUSTUP_TOOLCHAIN lets rustup-managed cargo switch toolchains if the image carries multiple ones.
  5. Resolves bldopt_manifest_path against the cwd's git top-level (via git rev-parse --show-toplevel) so verify works from anywhere inside the checkout.
  6. Hashes every rebuilt artifact and looks for a match against the original. Prints ✅ on match (with the matching crate's name); ⚠ + non-zero exit on mismatch (with each rebuilt artifact's name + hash).

The user is responsible for checking out the matching commit before running verify; verify rebuilds from the working tree. (source_repo and source_rev are embedded in meta to help users find the right commit, but verify itself doesn't clone — that would add a separate trust path.)

End-to-end example

$ stellar contract build --backend docker
ℹ Pulling from stellar/stellar-cli
   Digest: sha256:cb2fc3116a6ace37a77ca6bb88afb4bee57fc746cd556a4373f2c3ee95d4e917
   Status: Image is up to date for stellar/stellar-cli@sha256:cb2fc3...
ℹ contract build --manifest-path /source/contracts/foo/Cargo.toml --profile release --locked --meta bldimg=docker.io/stellar/stellar-cli@sha256:cb2fc3...
   Compiling foo v…
    Finished `release` profile [optimized] target(s) in 1.09s
ℹ Build Summary:
   Wasm File: target/wasm32v1-none/release/foo.wasm (907 bytes)
   Wasm Hash: 9f86d081…
✅ Build Complete

$ stellar contract info meta --wasm target/wasm32v1-none/release/foo.wasm
cliver=26.0.0#abc1234
bldimg=docker.io/stellar/stellar-cli@sha256:cb2fc3...
rsver=1.83.0
source_repo=https://github.com/user/my-contract
source_rev=abc1234567890abcdef…
bldopt_manifest_path=contracts/foo/Cargo.toml
bldopt_package=foo
bldopt_profile=release

# Later, on a different machine, with the matching commit checked out:
$ stellar contract build verify --wasm-hash <hash> --network mainnet
ℹ Loading contract from network...
ℹ Loading meta from contract...
   Original wasm hash: 9f86d081…
   stellar-cli version: 26.0.0#abc1234
   rust version: 1.83.0
   Docker image: docker.io/stellar/stellar-cli@sha256:cb2fc3...
   Manifest path: contracts/foo/Cargo.toml
   Package: foo
   Profile: release
ℹ contract build --manifest-path /source/contracts/foo/Cargo.toml --profile release --locked --meta bldimg=docker.io/stellar/stellar-cli@sha256:cb2fc3...
   Compiling foo v…
✅ Build Complete
✅ Verified: rebuilt foo wasm matches 9f86d081…

The host CLI's version is irrelevant for verifying a docker-built wasm — whatever cli is in the image is what built (and rebuilds) the wasm.

Notes

  • Communication with the daemon: bollard's HTTP API over the docker socket (/var/run/docker.sock, or whatever --docker-host / DOCKER_HOST points at). Same connect_to_docker helper used by stellar container start/stop/logs, with the same Docker Desktop fallback ($HOME/.docker/run/docker.sock). No shell-out to the docker CLI. A podman socket exposing the Docker API would also work (untested).
  • Default image is digest-pinned: --backend docker (no =...) defaults to docker.io/stellar/stellar-cli@sha256:cb2fc3..., not stellar/stellar-cli:latest. Recording a digest immediately makes builds reproducible day one and avoids the Apple Silicon RepoDigests-after-cross-platform-pull quirk. Bumping the default is a single-line const change in build.rs (see comments there for the recipe). Users who want a different image specify --backend docker=....
  • Entrypoint override: the official stellar/stellar-cli image's entrypoint runs entrypoint.sh, which launches dbus + gnome-keyring. That setup fails when the container runs as a host UID without an /etc/passwd entry — see Docker image's entrypoint dbus init fails when run as non-root UID #2543. We override the entrypoint to point straight at the stellar binary, which is fine because contract build doesn't touch the keyring.
  • Caching: the bind-mount of host ~/.cargo/registry lets the container reuse crate downloads the host already has.
  • Wasm target installation: deferred to the image. The official image has wasm32v1-none pre-installed for its default toolchain; if RUSTUP_TOOLCHAIN selects a different one (verify on a wasm built with another rust version), the cli/cargo handle target installation themselves.
  • Toolchain pinning: verify sets RUSTUP_TOOLCHAIN=<rsver> inside the container (and cargo +<rsver> for local rebuilds) so the rust version matches whatever the original build used.
  • Image fully-qualified: bldimg is normalized to <registry>/<path>@sha256:<digest> (e.g. stellar/stellar-cli:latestdocker.io/stellar/stellar-cli@sha256:…) so verify can resolve it without relying on the local registry config.
  • Source URL canonicalization: source_repo is normalized to https://… form (e.g. git@github.com:user/repo.githttps://github.com/user/repo).
  • Build options auto-recorded: bldopt_manifest_path is recorded relative to the git repo root regardless of whether --manifest-path was passed on the CLI. Verify resolves it against the cwd's git top-level so the command works from anywhere inside the checkout.
  • No new in-container flags: the host invokes stellar contract build inside the image with only flags that exist in published stellar/stellar-cli images today (--manifest-path, --profile, --locked, --meta, --package, --features, --all-features, --no-default-features, --optimize). bldimg is forwarded via --meta bldimg=<digest>, not a new flag.
  • No bldbkd field: presence of bldimg is the only signal needed to distinguish a docker build from a local one.
  • Aborted container runs: may leave a stopped container; clean with docker container prune.

Performance/runtime caveats

Building inside an amd64 container on a non-amd64 host (Apple Silicon, Linux/arm64) runs under emulation. For small contracts the difference is negligible; for workspaces with heavy dep trees the emulated build can be substantially slower than a native host build. Container runtimes that don't ship qemu/binfmt support won't run amd64 containers on arm64 hosts at all. See #2506 (comment).

Related issues

Status

This is an experiment in validating the ideas in #2506. May or may not be destined for merging — at this moment it's an experiment in validating the approach.

@github-project-automation github-project-automation Bot moved this to Backlog (Not Ready) in DevX Apr 27, 2026
@leighmcculloch
Copy link
Copy Markdown
Member Author

Each meta field needs a very detailed specification of exactly what valid values are expected, and the format(s) those values can take.

For example, depending on how you install the stellar-cli today, the cliver field can contain any of the following formats:

Homebrew:

cliver: 26.0.0#

Cargo crates.io install:

cliver: 26.0.0#60f7458e7ecffddf2f2d91dc6d0d2db4fab03ecc

Cargo git install:

cliver: 26.0.0#v20.0.0-836-gfe07b3678833e07c43235a6caaeccff81e146856

Without a precise spec for each field, downstream tooling (verifiers, registries, indexers) can't reliably parse or validate these values.

@leighmcculloch
Copy link
Copy Markdown
Member Author

Currently only the cargo build step runs inside the container. The image is a stock rust:latest (with the wasm target installed at runtime) — everything the stellar-cli does after the build (meta injection, post-processing, etc.) runs on the host as part of the stellar binary, against the wasm output that comes back through the bind mount.

That works, and the host-side stellar-cli version is captured in the wasm via the cliver meta, so verify does have what it needs to detect a mismatch. But it still splits the build pipeline across two environments: a pinned, reproducible one (the image digest) and a host-resolved one (the user's installed stellar). To reproduce a build, a verifier needs both the right image and the right stellar on their host — the image digest alone isn't sufficient. To verify a build produced by a different stellar version than the one installed locally, the user has to install that other version first.

A way to get the best of both:

Embed a Dockerfile in the stellar-cli that, at build time, layers on top of whatever rust base image is requested:

  1. FROM the user-chosen rust base image (still flexible — users can pin to any rust:<sha> they trust)
  2. Add the wasm target
  3. Install the requested stellar-cli version

Both the rust version and the stellar-cli version are specified at build (and verify) time, the image gets built locally, and the entire build pipeline runs inside it. The user on the host doesn't need to install a matching stellar-cli to verify a build produced by a different version — that version is installed into the image instead.

This avoids the supply-chain cost of us owning and publishing a bespoke stellar build image, keeps the rust base image flexible, and still lets the image digest capture the whole pipeline.

It also resolves the cliver-format issue raised in #2525 (comment) — since the embedded Dockerfile installs the stellar-cli exactly one way, cliver will be rendered exactly one way too, instead of varying by host install method (homebrew vs. crates.io vs. cargo-git).

It also keeps the door open for SDF or someone else to host prebuilt images later as a convenience — but we don't have to figure that out for the first iteration. The embedded-Dockerfile approach defers that decision without foreclosing it.

@leighmcculloch
Copy link
Copy Markdown
Member Author

Opened an issue about the cliver inconsistency here:

@leighmcculloch

This comment was marked as outdated.

@leighmcculloch

This comment was marked as outdated.

@leighmcculloch
Copy link
Copy Markdown
Member Author

leighmcculloch commented May 1, 2026

Opened an issue about dbus creating problems with using the image for the build for the verification step:

@fnando
Copy link
Copy Markdown
Member

fnando commented May 1, 2026

@leighmcculloch I just tried this, but I'm getting a warning, even though there are no unstaged files.

$ git status
On branch main
nothing to commit, working tree clean

$ stellar contract build --backend docker
⚠️ git working tree has uncommitted changes; source_repo/source_rev/bldopt_* not embedded in contract metadata. Commit changes for a reproducible build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog (Not Ready)

Development

Successfully merging this pull request may close these issues.

Add --docker option and stellar contract verify for reproducible builds

2 participants