Skip to content

Fidelity report

waznggo edited this page Jul 26, 2026 · 1 revision

The fidelity report

The hard question about any local reproduction is "it passed here — will it still fail on CI?" So reenact states, before you ask, every way this run differs from the real one.

  Fidelity
    ✓ Commit         ca51233 — the run's exact commit is checked out
    · Image          ghcr.io/catthehacker/ubuntu:act-24.04 — the closest community image
    ⚠ Event          pull_request, only partially reconstructed (13 fields)
    ✗ Upstream jobs  this job needs build and downloads an artifact from them
meaning
same as CI
· different, but not in a way that changes behaviour
may change behaviour
will change behaviour — read it before you trust the result

Two rules decide what appears here, and both came out of measurement:

Findings are scoped to the steps that actually run — index 0 through the failing step, minus steps CI skipped. In the corpus, 20 of 56 jobs mention a secret somewhere in the workflow file, but only 3 mention one in the failing job, and in all 3 it sits after the breakpoint. A file-wide scan would have cried wolf twenty times.

is a provable claim, not a strong opinion. Only two things earn it: a step that downloads an artifact from a job that will not run (so the artifact does not exist), and a called workflow that demonstrably changed since the run.

The findings you will actually see

Image

GitHub does not publish its runner images. ghcr.io/catthehacker/ubuntu is the closest community equivalent, and preinstalled tool versions differ. If a tool is missing, retry with --image full.

Cache

CI restored a cache; your first local run starts cold and builds from scratch. act ships its own cache server, so a second run of the same job is warm. A failure that only reproduces with CI's cached contents will not show up on the first run.

Upstream jobs (needs)

Only the failing job runs. Anything its dependencies produced — outputs, artifacts — is absent. When a live step downloads an artifact from those jobs, this becomes : that step cannot succeed.

Secrets

Secret values cannot be read from the GitHub API by design. reenact detects which secrets the running steps reference and reports the ones you have not supplied. Pass them with --secret NAME=value or --secret-file path; they go into a 0600 file, never onto the command line where ps would show them, and are deleted when you exit.

GITHUB_TOKEN is reported separately: it is deliberately not passed into the container.

Event

push payloads reconstruct almost completely; pull_request ones only partially. Steps reading github.event.* may see empty values.

Called workflow

Only for jobs that call another workflow.

  • the call is same-repo or pinned to a SHA — the file read is the file the run used.
  • the call uses a moving ref (@main). GitHub does not record which commit the run used, so what you get is today's file.
  • proof that it changed: a step from the run no longer exists in the file, named in the finding. Measured example: charmbracelet/meta bumped a pinned actions/setup-go SHA after the run, so the local run uses a different version of that action than CI did.

Ignored keys

act does not implement timeout-minutes, concurrency, permissions, environment and a few others. They are dropped from the generated workflow and listed here rather than silently ignored.

Clone this wiki locally