Skip to content

Restow 0.4.3

Choose a tag to compare

View all tags
@github-actions github-actions released this 22 Oct 09:36
· 162 commits to main since this release
v0.4.3
64762b2

Restow 0.4.3. Released 2026-10-22, channel beta.

Beta release. Run it alongside your existing backups, not as your only one, until
you have verified restores against your own data.

Summary

Restow 0.4.3 stops a locked repository from failing a Proxmox VE restore check (it is
postponed and, if the lock stays, reads "Could not check" in amber with who holds the lock)
and tidies VMs & containers up into tabs with search, filters, a detail panel and tables
that no longer jump while backups run. Proxmox VE nodes and agents follow a server update
even when its announcement went unanswered, "Update nodes and agents now" starts that step
by hand, and restow-pve restarts the PVE daemons after it updates itself. No database
migration; nodes still on restow-pve 0.4.1 need one update by hand (Upgrade Notes).

Breaking Changes

None. The integration API contract stays at 1.5.0. In the session API, a Proxmox VE guest's
lastVerify.state can be yellow (with notChecked) and a node carries offeredVersion
and outdated; both are additive.

Added

  • Updates tab: Update nodes and agents now on the card of nodes and agents. When the
    server runs a newer release than its Proxmox VE nodes and agents but no update led into the
    step where they follow it (an update by hand, or one whose announcement went unanswered),
    an administrator with every tenant starts that step: new runs pause for at most 15 minutes,
    reachable nodes and agents install the release when idle, and the pause ends once they are
    done (POST /api/v1/updates/fleet/update-now, audited as update.fleet_started with
    manual: true). Outside an update, nodes and agents that are behind show as "Update
    available".
  • Proxmox VE inventory (GET /api/v1/pve): every node carries offeredVersion (the
    restow-pve release the server offers) and outdated next to its helperVersion.
  • Releases publish the two images of the public demo as well (restow-demo-seed and
    restow-demo-dovecot, amd64 and arm64, signed, with SBOMs and the release's tags), and
    deploy/demo/reset.sh can follow a channel or a fixed version (RESTOW_DEMO_FOLLOW in
    deploy/demo/.env): the nightly reset pulls the images of one release and stops before it
    changes anything when they do not match or a pull fails (deploy/demo/README.md).

Changed

  • VMs & containers is tidied up into three tabs with counts: Guests (the default), Backup
    jobs and Nodes. The note that the Proxmox VE host itself is not backed up is one line on the
    Nodes tab and can be hidden. The Nodes tab shows each node's heartbeat, its restow-pve
    version with "Update to …" when the server offers a newer release (outdated), its Proxmox VE version, how many
    guests it has, its queue and its problems; queue control and revoking are in the row menu.
  • The guests table has a search over ID and name, filters by node, kind (VM or container),
    backup job (also "No backup job") and state, a "Problems only" button (failed, warnings or
    gaps, a restore check that failed or could not run, waiting for a disconnected or paused
    node, not backed up), sortable columns (ID, name, node, last backup, backup size, state), pages of
    25 to 250 guests and the count of what the filters leave. Tab, search, filters, order,
    page and the open guest live in the address, so a link opens the same view.
  • A guest's row shows one state badge, chosen by priority, with a small progress bar while
    its backup runs or "+N" for further problems. Clicking the row (or its name) opens a detail
    panel with every state, the complete error and warnings of the last run in a scrollable,
    copyable block, the restore check with what it found (or, when a locked repository kept it
    out, who holds the lock, amber), the backup sizes, the backup job, the
    newest runs and the actions (back up now, stop, withdraw, check now, open the guest page).
  • The Backup column says how large a guest's first (full) backup was and what the last
    follow-up added, for example "First: 32 GB on Oct 3 · Last: +210 MB"; the detail panel
    shows the new data of each of the last 30 backups as small bars (#58).
  • The README screenshots show 0.4.2, with two new views (the mail archive and a server).
  • Development: the web tests run on happy-dom 21.

Fixed

  • Proxmox VE: right after a container backup, the new restore check often failed with
    restic check failed (exit code 11) the "unlock" command can be used to remove stale locks
    and the guest showed a red "Restore check failed", although the check never ran: another
    run (usually the guest's own backup, whose lock a cancel or an API restart had left behind)
    held the repository's lock. A check a lock keeps out is no longer a failed check: it is
    tried again ten minutes later, the previous result standing, and after two hours the
    restore point reads "Could not check: repository locked" (yellow, attention, never red or
    green) with restic's report of who holds the lock. The worker removes stale restic locks
    before the check and before retention, restic waits up to two minutes for a lock during the
    check and up to 15 minutes during the container backup (restow-pve), a guest is not checked
    while a run of it is going or its node took on a backup of it, and a new restore point is
    checked two minutes after its commit at the earliest. A cancelled or ended run may still
    delete its own restic lock (every other request of it is refused and audited). A failed
    check keeps restic's explaining lines, not only its last one. No database migration.
  • Updates: announcing an update could report "The updater is not running" although the
    updater accepted it a few seconds later. The api waited 5 seconds for an answer while the
    updater checks Docker, the Compose configuration, the disk space and the running version
    first (5 to 12 seconds on real installations). The update then ran without the pause of new
    runs, without the update.scheduled audit entry and without the step in which Proxmox VE
    nodes and agents follow the server, so they stayed on the old release. The api now waits up
    to a minute for the announcement (30 seconds for cancelling, dismissing, restarts and
    deleting a dump); when the answer is still lost it reads the updater's state again and
    treats the update as accepted when the updater shows it. The pause is lifted only on a
    clear refusal or an idle updater, and "Start now" after waiting for runs no longer sends a
    false failure notification. The updater answers faster: it checks once per announcement
    and reuses the measured database and image sizes of the last ten minutes (the run's first
    step measures again). The Updates tab reads the state again after a failed announcement
    and says it is checking whether the update was accepted instead of "not running".
  • Updates: the Updates tab could say "The updater is not running" for a few seconds every now
    and then. When the updater's check of the last 30 seconds had expired, it checked again
    before it answered (several short-lived helper containers), which could take longer than
    the 5 seconds the api waits for the state, and the api then remembered "no updater" for
    8 seconds. The updater now answers at once with its last check and checks again in the
    background (an update still checks anew before it starts), and a plain state read that
    times out keeps the updater's last state for up to 30 seconds after its last answer.
  • Proxmox VE: after a self-update (automatic, "update now" or restow-pve update) the PVE
    daemons kept the old storage plugin loaded until they restarted. restow-pve now restarts
    pvedaemon, pveproxy, pvestatd and pvescheduler like the installer does (systemctl reload-or-restart on the running ones; a failure is logged and does not undo the update).
    restow-pve update also restarts the running restow-pve service, so the node reports the
    new release with its next heartbeat. A node that an older restow-pve updated restarts the
    daemons once by itself (it remembers the plugin files it restarted them for).
  • VMs & containers no longer jumps while backups run: the guest and job tables keep fixed
    column widths and one row height (loading rows included), long texts are cut to one line
    instead of wrapping, the progress has a reserved slot, and "Back up now", "Stop",
    "Withdraw" and "Stopping…" share one button size. Error texts no longer appear in red under
    a guest's row, and warnings no longer only in a tooltip.
  • Demo: during the nightly reset the demo shows a short page in English and German that
    reloads every minute, instead of a 502. After signing in, the overview opens on a customer
    tenant instead of the empty provider organization, and the simulated machines report the
    running release as their agent version instead of 0.2.1.

Security

None.

Upgrade Notes

New image, no database migration, no new required environment variables, and the
docker-compose.yml of 0.4.0 stays.

  • Proxmox VE nodes should run restow-pve 0.4.3: it waits for the server's repository lock
    during a container backup and restarts the PVE daemons after it updates itself. Nodes on
    restow-pve 0.4.2 install it with the step after the server update (or "Update nodes and
    agents now"). Nodes on restow-pve 0.4.1 ignore "update now" and that step; they update with
    their own check (two minutes after the service starts, then every six hours), or by hand
    once no backup or restore runs on the node:
    /opt/restow-pve/bin/restow-pve update, then
    systemctl restart restow-pve pvedaemon pvestatd pveproxy pvescheduler. From 0.4.3 on
    restow-pve restarts the daemons itself; a node that an older release updated does it once
    after the new release starts.
  • Restore checks that a locked repository failed before the update stay red until the next
    check of the guest ("Check now" on the guest or VMs & containers).
  • Demo hosts: the demo images are now published with every release; to let the demo follow
    them, set RESTOW_DEMO_FOLLOW as deploy/demo/README.md describes.
  • Rollback: 0.4.2 starts on the same database. It shows a restore point that could not be
    checked as unchecked.

Known Issues

  • The new parts were tested against a simulated Proxmox VE API and S3 store only, never
    against a real Proxmox VE cluster or S3 provider.
  • The redesigned VMs & containers page was tested with DOM tests, not visually in a browser
    with thousands of guests.
  • The known issues of 0.4.2 still apply, except the file share repository healing, which
    comes later.

Verification

Before the tag, on the release branch (main plus the 0.4.3 workstreams): lint and typecheck
of all workspaces, the unit and Postgres suites of every package on Postgres 16 (with the
Postgres guard), the web DOM tests on happy-dom 21 including the layout-width and
green-is-proof checks, the v1 OpenAPI test, gofmt, go vet and the Go tests of the agents,
the Perl tests of the Proxmox VE plugin, the server installer and demo reset tests, and
shellcheck. The release smoke runs in the release pipeline; no real Proxmox VE cluster or S3
provider was used before the tag.

Release pipeline, 2026-10-22:

Check Result Duration
1. Install: images, compose up, migrations (new install and upgrade) PASS 87s
2. Health endpoints, worker and scheduler registered PASS 0s
3. Passkey sign-in E2E with a virtual authenticator, tenant creation, i18n de/en PASS 2m 15s
4. Microsoft 365 backup and restore against the dev tenant SKIPPED 0s
5. IMAP backup and restore against Dovecot, hash comparison PASS 8s
6. Journal receipt, chain verification, export PASS 3s
7. Standalone restore with restow-restore from the storage of check 5 PASS 24s
8. Storage targets: S3 (Garage), local path, mounted directory PASS 20s
9. Image scan (Trivy) and dependency audit (pnpm audit) PASS 21s
10. Endpoint backup: the agent enrolls, backs up a folder, restores it PASS 11s
11. Mail import (folder and upload) and export (EML ZIP, MBOX), compared by hash PASS 8s
12. File share backup against Samba: back up, browse, ZIP, restore with permissions, copy job PASS 2m 17s

11 of 12 checks passed in full, 0 partly, 1 skipped, 0 failed. The full report, including the reason for every skipped or partial check, is attached to the release as smoke-report.md.

The Community images were smoke-tested on their own (smoke-report-community.md, smoke-report-community-arm64.md): amd64 PASS with gaps: 9 of 12 checks passed in full, 0 partly, 3 skipped (see the reasons below); arm64 PASS with gaps: 9 of 12 checks passed in full, 0 partly, 3 skipped (see the reasons below).

Images

Two builds of the same release. The full build is the Apache-2.0 core plus the Business and Service Provider modules (Restow Enterprise License), which stay locked until a license key is installed. The Community build is the Apache-2.0 core alone.

ghcr.io/restow-backup/restow:0.4.3
ghcr.io/restow-backup/restow-web:0.4.3
ghcr.io/restow-backup/restow-community:0.4.3
ghcr.io/restow-backup/restow-web-community:0.4.3

The digests of the multi-arch manifests (the opt-in updater verifies the pulled images against the two lines of its build):

restow: sha256:a718d37717583ff476e388d8065c6b8688e8954d899c5ec7c26096cf1eb90d6a
restow-web: sha256:55de05bd171f57844dd3925032b0f352030fa3c649cca8a5026fb29783e40e1f
restow-community: sha256:2e30dc359e3ba25f6d61e9bf4988127c06bd62f88989d92b0b504b68534e730b
restow-web-community: sha256:d11156bab53a9726bf4a29ffea17a76ce149f3cba807bb4b0d4ae5b76762114c

All four images are multi-arch (linux/amd64, linux/arm64), signed with cosign (keyless) and carry an SPDX SBOM as attestation. Verify any of them with:

cosign verify ghcr.io/restow-backup/restow:0.4.3 \
  --certificate-identity-regexp '^https://github.com/restow-backup/restow/\.github/workflows/release\.yml@refs/tags/v' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
cosign verify ghcr.io/restow-backup/restow-web:0.4.3 \
  --certificate-identity-regexp '^https://github.com/restow-backup/restow/\.github/workflows/release\.yml@refs/tags/v' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
cosign verify ghcr.io/restow-backup/restow-community:0.4.3 \
  --certificate-identity-regexp '^https://github.com/restow-backup/restow/\.github/workflows/release\.yml@refs/tags/v' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com
cosign verify ghcr.io/restow-backup/restow-web-community:0.4.3 \
  --certificate-identity-regexp '^https://github.com/restow-backup/restow/\.github/workflows/release\.yml@refs/tags/v' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

The public demo's own images, ghcr.io/restow-backup/restow-demo-seed:0.4.3 and ghcr.io/restow-backup/restow-demo-dovecot:0.4.3 (deploy/demo), carry the same tags, signatures and SBOMs; they are no part of an installation.

docker-compose.yml and env.example are the release stack (deploy/release/); install.sh installs it on a dedicated Linux VM (README.md, "Install with the script"; install.sh.sha256 for sha256sum -c after the download); restow-agent-THIRD_PARTY_NOTICES.txt holds the licenses of the agent binaries and of the restic they run; SHA256SUMS lists every file of this release and is signed (SHA256SUMS.sigstore.json).

Assets 31
Loading