Skip to content

Troubleshooting

Jump to bottom
tej edited this page Jun 2, 2026 · 4 revisions

Troubleshooting

The failures you are most likely to hit, with the actual fix.

Build: btrfs/version.h: No such file or directory 

# go.podman.io/storage/drivers/btrfs
fatal error: btrfs/version.h: No such file or directory
# github.com/proglottis/gpgme
Package gpgme was not found in the pkg-config search path.

You built without the tags. Use make build, or pass -tags "containers_image_openpgp exclude_graphdriver_btrfs exclude_graphdriver_devicemapper". Full rationale in Building.

go test: -tags space-separated list contains comma

You combined the tag set with integration using a comma. The tag list can't mix spaces and commas — append with a space: -tags "$(TAGS) integration".

CI job fails in ~28s: executable file 'node' not found in $PATH

The runner runs the job inside a container, and actions/checkout/actions/cache are JavaScript actions needing a node binary the base image doesn't ship. Install nodejs in a run: step before any uses: step (run steps use the shell, not node).

CI: runs stuck in waiting, no jobs assigned

runs-on: names a label no registered runner carries. This instance's runner is labelled self-hosted.

CI: integration job fails immediately on --privileged

The podman-in-podman job needs a privileged container for nested podman (storage + play kube hostPort networking). If the runner forbids privileged jobs, that has to be allowed in the runner config — it can't be fixed in the workflow.

Apply returns 502

The registry was unreachable (or the tag doesn't exist) during the mandatory pre-pull. The message is the registry's own. No secret or pod is created — fix the image ref / registry and retry. ?skip_pull=true bypasses the pre-pull only when the image is already local.

A deleted instance left its secret behind

DELETE keeps per-instance secrets unless you pass ?prune_secrets=true (and ?prune_volumes=true for volumes). To reap an already-orphaned secret, call DELETE ...?prune_secrets=true&prune_volumes=true again — delete is an idempotent reconcile and will remove orphans even though the pod is gone. Or use the per-resource endpoints: DELETE /hosts/{host}/secrets/{name} and DELETE /hosts/{host}/volumes/{name}.

Host healthz fails / unknown host

  • unknown host → no hosts/<id>.yaml with that id, or it didn't load. Check -hosts-dir.
  • SSH/tunnel errors → wrong addr, wrong ssh_key, the key isn't authorised on the target, or the socket path is wrong. Verify with ssh <addr> true and check the socket path is /run/user/<uid>/podman/podman.sock. See Provisioning a Podman Host.

play kube fails: catatonit: executable file not found

The target (or your local dev box) lacks the pause binary podman uses for the pod infra container. Install it: apt-get install catatonit (Debian/Ubuntu). The quay.io/podman/stable CI image already includes it.

A bad keys.yaml edit — am I locked out?

No. On SIGHUP, a parse error or an empty key list is logged and the previous list stays live. Fix the file and reload again. See Operating.

Related

Clone this wiki locally