Backup Restore and Updates

Backup, Restore, and Updates

Pizarra treats backup, restore, and endpoint rollout as guarded operational workflows. They reduce avoidable partial failure; they do not replace encrypted storage, service supervision, or operator review.

Create a hot backup

tiza backup /srv/secure-backups/pizarra
tiza backup verify /srv/secure-backups/pizarra/pizarra-<timestamp>

With no destination, the hub writes below <store>/backups. A selected destination inside a Git worktree is rejected unless --force is used. Do not make force routine policy.

The hub:

• uses the SQLite online-backup API for all three databases;
• holds one ordered snapshot barrier across configuration, workflows, tasks, messages, and SQLite so every included piece represents one point in time;
• copies pizarra.conf and, when present, exactly messages.jsonl, state.json, tareas.json, and workflows.json;
• records sizes and SHA-256 values in MANIFEST;
• assembles in a temporary directory;
• publishes only by final rename; and
• applies owner-only permissions.

A backup is a credential bundle

Hub configuration and live credentials are included by design.

• encrypt backup storage;
• limit access to recovery operators;
• keep it outside the repository;
• keep it outside pzweb static assets;
• keep it outside shared exchange directories;
• verify after creation; and
• rehearse restore on an isolated host.

Verification is local and non-mutating. It checks every manifest size and hash, parses the saved configuration with the hub's real loader, runs SQLite integrity and foreign-key checks on all three databases, requires the sqlite-v1 authority marker, and rejects missing, duplicated, or unlisted payload—including an undeclared SQLite WAL/shared-memory file. An isolated restore remains the strongest end-to-end exercise.

Important exclusions

Built-in backup and restore do not include:

• wfhistory/ workflow undo snapshots;
• legacy appdocs/ migration files; or
• /var/lib/pizarra/releases/ endpoint artifacts; or
• any content below the configured external shared/NFS root.

Current application manuals are stored in the SQLite registry after migration, but old migration inputs may still matter. Release artifacts can normally be regenerated from their exact committed source. pizarra.conf is included, so the shared-path reference is present, but the NFS bytes are not. If an excluded directory must be retained, capture it separately while writers are quiescent or through a coordinated filesystem/NFS snapshot. Protect and verify it under the same security policy.

Dry-run restore

Restore is a local operation on the hub host. Stop the exact hub first:

sudo systemctl stop pizarra.service
PIZARRA_CONF=/etc/pizarra/pizarra.conf \
  tiza restore /srv/secure-backups/pizarra/pizarra-<timestamp> --dry-run

Review the displayed:

• active configuration path;
• target store path;
• selected backup;
• service reachability result; and
• credential comparison warning.

Confirm restore

PIZARRA_CONF=/etc/pizarra/pizarra.conf \
  tiza restore /srv/secure-backups/pizarra/pizarra-<timestamp> --confirm

The command:

• acquires the same persistent .pizarra-hub.lock lifetime lock as the hub and also refuses while the target hub answers;
• verifies the backup before mutation;
• requires explicit confirmation;
• moves current state into a timestamped .pre-restore-* safety directory;
• moves SQLite sidecars with old state; and
• stops before replacement when saved and current credentials differ or cannot be compared.

The lock closes the startup race before the listener exists. Its file must already be a safe mode-0600 regular file; run the new hub once with --migrate-only before restore if old state has no lock metadata. The lock file is persistent metadata, not a stale PID file, so do not delete it during cleanup. Restore also snapshots root names before moving them and proves no residual file remains before installing the allow-listed payload.

When a privileged recovery restores a store owned by a dedicated hub account, replacement files retain the existing store/config owners rather than becoming unreadable root-owned mode-0600 files.

--force is separate acceptance of a fleet-wide credential rollback consequence, not a generic convenience switch.

Restore can replace pizarra.conf and therefore its [shared] dir reference; it never changes shared/NFS content. After restore, verify that this reference and pzweb [web] shared remain exactly equal and select the intended mount. Recover shared data only from its separately coordinated snapshot.

After restore:

sudo systemctl start pizarra.service
tiza team list
tiza task list all
tiza wf list
tiza inbox --all
tiza fleet

Also verify application registries, remote delivery, pzweb startup, and recent messages. Retain the pre-restore directory until validation finishes.

Prepare endpoint releases

Build from a clean committed tree:

./configure
make test
make publish

Run make publish as the account that owns the release root. RELEASEDIR defaults to /var/lib/pizarra/releases; overrides must be absolute. Configure [server] releases to that exact path. The loader consumes it literally, so a relative value is unsafe because it depends on the hub's working directory. Keep the root real, private at mode 0700, and outside the source repository.

Publication stages tiza-<os>-<cpu> and src.tar.gz under temporary names, renames them atomically, and writes VERSION last. The hub reads those exact names and offers an update only when the published version exactly matches its running version.

Configure the hub's release root, inspect the fleet, and roll out one host:

tiza fleet
tiza update reviewer
tiza fleet

Then expand deliberately:

tiza update all

The caller selects a team or all; it cannot supply an arbitrary download URL.

Endpoint installation safety

The endpoint:

1. downloads the advertised size and SHA-256 in chunks;
2. rejects excess size, incomplete data, or a wrong digest;
3. stages the candidate;
4. checks its reported version;
5. creates a rollback copy;
6. replaces atomically;
7. checks the installed binary; and
8. restores the copy if the final check fails.

A verified success removes rollback copies.

Update boundary

• The built-in channel updates tiza endpoint daemons only.
• Hub, pzweb, assets, configuration, and service units use normal operator-controlled installation.
• Local teams use the control host's installed binary.
• Hash verification is integrity protection, not independent release signing.
• Source fallback requires a compiler on the endpoint.
• autoupdate = on is an explicit change-control decision.

Keep automatic update off when independent release approval is required.

Recommended recovery drill

1. Create and verify a fresh backup.
2. Copy it through the protected backup channel.
3. Restore into a disposable host and store.
4. Start the isolated hub on non-production loopback ports.
5. Verify registries, journal, inbox cursors, tasks, and workflows.
6. Test one endpoint delivery with disposable credentials.
7. Record exclusions and independently restored auxiliary history.
8. Destroy only the exact disposable environment.

See Operations Runbook, Security Model, and the canonical operations documentation.