Description
NS8 backups currently require internet access even when the restic backup image is already cached locally. core/imageroot/var/lib/nethserver/node/update-core.d/96cleanup_restic_images unconditionally removes cached restic image tags on every core update, and prepare_restic_command() (agent/pypkg/agent/__init__.py:215-271) never passes --pull=missing (or similar) to podman. So every backup run tries to re-pull ghcr.io/nethserver/restic:<version> first.
In air-gapped or intermittently-offline clusters this means backups either hang for minutes (3 retries against an unreachable registry) or fail outright, even though a working local image is available.
We want backups to work offline when a compatible restic image is already present locally, using it instead of always trying to re-pull.
Backup related logs:
Jul 07 16:06:59 ns8 agent@node[3314501]: Trying to pull ghcr.io/nethserver/restic:3.20.0...
Jul 07 16:07:29 ns8 agent@node[3314501]: time="2026-07-07T16:07:29+02:00" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://ghcr.io/nethserver/restic:3.20.0: pinging container registry ghcr.io: Get \"https://ghcr.io/v2/\": dial tcp 140.82.121.33:443: i/o timeout"
Jul 07 16:08:00 ns8 agent@node[3314501]: time="2026-07-07T16:08:00+02:00" level=warning msg="Failed, retrying in 1s ... (2/3). Error: ...same..."
Jul 07 16:08:31 ns8 agent@node[3314501]: time="2026-07-07T16:08:31+02:00" level=warning msg="Failed, retrying in 1s ... (3/3). Error: ...same..."
Jul 07 16:09:03 ns8 podman[924198]: ... image pull-error ghcr.io/nethserver/restic:3.20.0 unable to copy from source docker://ghcr.io/nethserver/restic:3.20.0: ...i/o timeout
Jul 07 16:09:03 ns8 agent@node[3314501]: [ERROR] restic stats failed for 86d1a8ac-ef89-557a-8e19-8582ab86b7c4/traefik/e410ec36-ab9d-4eb9-9dc0-4e3ae4710825: Command '['podman', 'run', ...]' returned non-zero exit status 125.
Alternative solutions
- Use cached rootfull image: distribute rootfull restic image to all module agents.
- Pull policy fix only: pass
--pull=missing to the podman run invocation so a cached local image is used when present, and pull is only attempted when missing.
- Pull policy fix + safer cleanup: additionally stop
96cleanup_restic_images from removing the currently-in-use image tag, so a cached image always survives across updates for offline use.
References
Description
NS8 backups currently require internet access even when the
resticbackup image is already cached locally.core/imageroot/var/lib/nethserver/node/update-core.d/96cleanup_restic_imagesunconditionally removes cachedresticimage tags on every core update, andprepare_restic_command()(agent/pypkg/agent/__init__.py:215-271) never passes--pull=missing(or similar) to podman. So every backup run tries to re-pullghcr.io/nethserver/restic:<version>first.In air-gapped or intermittently-offline clusters this means backups either hang for minutes (3 retries against an unreachable registry) or fail outright, even though a working local image is available.
We want backups to work offline when a compatible
resticimage is already present locally, using it instead of always trying to re-pull.Backup related logs:
Alternative solutions
--pull=missingto thepodman runinvocation so a cached local image is used when present, and pull is only attempted when missing.96cleanup_restic_imagesfrom removing the currently-in-use image tag, so a cached image always survives across updates for offline use.References