Skip to content

Generalize Backup command #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mlaurense merged 4 commits into from
Aug 28, 2026
Merged

Generalize Backup command #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
877581e
Add --output-dir, --archive-name and --keep-dir to roll backup
marcepartment Aug 26, 2026
920f811
Fix infinite recursion in roll backup --help
marcepartment Aug 26, 2026
5e4aee8
Version bump
marcepartment Aug 26, 2026
d296282
Address review feedback on the backup output flags
marcepartment Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 108 additions & 24 deletions commands/backup.cmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,23 @@ BACKUP_NAME=""
BACKUP_DESCRIPTION=""
BACKUP_DUPLICATE_NAME="" # New environment name for duplication
BACKUP_DUPLICATE_DOMAIN="" # New domain for duplication
BACKUP_OUTPUT_DIR="" # Where the final archive is written (default: BACKUP_BASE_DIR)
BACKUP_ARCHIVE_NAME="" # Final archive basename, extension appended (default: backup_<env>_<id>)
BACKUP_KEEP_DIR=0 # Keep the uncompressed backup directory alongside the archive
BACKUP_OUTPUT_REDIRECTED=0 # Set when --output-dir is given, whatever path it names
PROGRESS=1

## Volumes are always staged inside the project: that is where the disk space is budgeted, and
## retention cleanup must never be pointed at a shared drop directory holding other projects'
## archives. --output-dir only moves the finished archive.
##
## Keyed on the working directory rather than ROLL_ENV_PATH, so `roll backup` from a project
## SUB-directory stages into <cwd>/.roll/backups. That is pre-existing behaviour and restore.cmd
## resolves the same way, so the two agree; changing it here alone would make backups land where
## restore does not look. Fix all of backup.cmd, restore.cmd and restore-full.cmd together, and
## note that restore-full.cmd has no ROLL_ENV_PATH at all — it never loads the env config.
BACKUP_BASE_DIR="$(pwd)/.roll/backups"

# Parse command line arguments
POSITIONAL_ARGS=()
# Start with any arguments passed from the main roll script
Expand All @@ -32,8 +47,11 @@ fi
while [[ $# -gt 0 ]]; do
case "$1" in
--help|-h)
roll backup --help
exit 0
## Do NOT re-invoke `roll backup --help` here. `backup` is on roll's ROLL_CMD_ANYARGS
## list, so roll's own parser stops at --help and passes it straight through to this
## script — re-invoking roll lands right back on this branch and recurses forever.
## usage.cmd renders ROLL_CMD_HELP (backup.help) and exits on its own.
source "${ROLL_DIR}/commands/usage.cmd"
Comment on lines +50 to +54
;;
--compression=*)
BACKUP_COMPRESSION="${1#*=}"
Expand Down Expand Up @@ -99,6 +117,19 @@ while [[ $# -gt 0 ]]; do
BACKUP_DUPLICATE_DOMAIN="${1#*=}"
shift
;;
--output-dir=*)
BACKUP_OUTPUT_DIR="${1#*=}"
BACKUP_OUTPUT_REDIRECTED=1
shift
;;
Comment on lines +120 to +124
--archive-name=*)
BACKUP_ARCHIVE_NAME="${1#*=}"
shift
;;
--keep-dir)
BACKUP_KEEP_DIR=1
shift
;;
--no-progress)
PROGRESS=0
shift
Expand Down Expand Up @@ -266,7 +297,7 @@ function detectEnabledServices() {

function createBackupDirectory() {
local timestamp=${1:-$(date +%s)}
local backup_dir="$(pwd)/.roll/backups/$timestamp"
local backup_dir="${BACKUP_BASE_DIR}/$timestamp"

# Create backup directories
mkdir -p "$backup_dir"/{volumes,config,metadata,logs}
Expand Down Expand Up @@ -632,8 +663,39 @@ function verifyBackup() {
fi
}

## Resolve and validate the archive destination up front. This deliberately runs before
## `env down` and before a single volume is tarred: a drop directory that turns out to be
## missing or read-only after a multi-hour backup means the whole run is thrown away.
function resolveBackupOutputDir() {
## --archive-name names a file inside the output directory, so a path separator in it would
## silently write somewhere else entirely (`--archive-name=../x`) and skip the checks below.
if [[ "$BACKUP_ARCHIVE_NAME" == */* ]]; then
error "--archive-name must be a filename, not a path: $BACKUP_ARCHIVE_NAME"
exit 1
fi

if [[ -z "$BACKUP_OUTPUT_DIR" ]]; then
BACKUP_OUTPUT_DIR="${BACKUP_BASE_DIR}"
return 0
fi

if ! mkdir -p "$BACKUP_OUTPUT_DIR" 2>/dev/null; then
error "Cannot create backup output directory: $BACKUP_OUTPUT_DIR"
exit 1
fi

if [[ ! -w "$BACKUP_OUTPUT_DIR" ]]; then
error "Backup output directory is not writable: $BACKUP_OUTPUT_DIR"
exit 1
fi

## Normalise to an absolute path: the final archive is written from inside a `cd` subshell,
## where a relative destination would resolve against the staging directory instead.
BACKUP_OUTPUT_DIR="$(cd "$BACKUP_OUTPUT_DIR" && pwd)"
}

function cleanupOldBackups() {
local backup_base_dir="$(pwd)/.roll/backups"
local backup_base_dir="${BACKUP_BASE_DIR}"

if [[ $BACKUP_RETENTION_DAYS -le 0 ]]; then
return 0
Expand All @@ -657,6 +719,7 @@ function performBackup() {

# Validate inputs
validateCompression || exit 1
resolveBackupOutputDir

# Handle interactive password prompt if needed
if [[ "$BACKUP_ENCRYPT" == "PROMPT" ]]; then
Expand Down Expand Up @@ -802,32 +865,53 @@ function performBackup() {
verifyBackup "$backup_dir"

# Create compressed archive for the entire backup
local archive_name="backup_${ROLL_ENV_NAME}_${timestamp}$(getCompressionExtension)"
local archive_name="${BACKUP_ARCHIVE_NAME:-backup_${ROLL_ENV_NAME}_${timestamp}}$(getCompressionExtension)"
local archive_path="${BACKUP_OUTPUT_DIR}/${archive_name}"
logMessage INFO "Creating final backup archive: $archive_name"
Comment on lines 867 to 870

# Suppress tar warnings when using --output-id

## The `cd` scopes tar's member paths to "<id>/..." so the archive extracts straight into a
## backups directory; the redirect uses the absolute destination so it can land elsewhere.
##
## pipefail is essential and is NOT set globally in roll: without it the pipeline reports the
## compressor's status, and gzip happily exits 0 on a truncated stream from a tar that ran out
## of disk. That would report success and then delete the only uncompressed copy below.
local archive_status=0
if [[ $BACKUP_OUTPUT_ID -eq 1 ]]; then
(cd "$(pwd)/.roll/backups" && tar -cf - "$timestamp" 2>/dev/null | $(getCompressionCommand) > "$archive_name")
(set -o pipefail; cd "${BACKUP_BASE_DIR}" && tar -cf - "$timestamp" 2>/dev/null | $(getCompressionCommand) > "$archive_path") || archive_status=$?
else
(cd "$(pwd)/.roll/backups" && tar -cf - "$timestamp" | $(getCompressionCommand) > "$archive_name")
(set -o pipefail; cd "${BACKUP_BASE_DIR}" && tar -cf - "$timestamp" | $(getCompressionCommand) > "$archive_path") || archive_status=$?
fi
Comment on lines 879 to 883

if [[ $? -eq 0 ]]; then
# Update latest symlink
(cd "$(pwd)/.roll/backups" && ln -sf "$archive_name" "latest$(getCompressionExtension)")


if [[ $archive_status -eq 0 ]]; then
## Only maintain the "latest" pointer when the archive was not redirected. A shared drop
## directory holds archives for many environments, where a single `latest` symlink would
## be overwritten by whichever project finished last. Keyed on whether --output-dir was
## given rather than on comparing the two paths, which differ as strings for the same
## directory once symlinks or a relative argument are involved.
if [[ $BACKUP_OUTPUT_REDIRECTED -eq 0 ]]; then
(cd "${BACKUP_BASE_DIR}" && ln -sf "$archive_name" "latest$(getCompressionExtension)")
fi

if [[ $BACKUP_OUTPUT_ID -eq 1 ]]; then
# Only output the backup ID for programmatic use
echo "$timestamp"
else
logMessage SUCCESS "Backup completed successfully!"
logMessage INFO "Backup ID: $timestamp"
logMessage INFO "Archive: $archive_name ($(du -h "$(pwd)/.roll/backups/$archive_name" | cut -f1))"
logMessage INFO "Location: $(pwd)/.roll/backups/"
logMessage INFO "Archive: $archive_name ($(du -h "$archive_path" | cut -f1))"
logMessage INFO "Location: ${BACKUP_OUTPUT_DIR}/"
fi


## Clean up the directory version, which the archive already contains. --keep-dir
## retains it for a caller that needs a restorable backup left in place — restore
## reads the directory form, so archiving the workspace after a backup only yields a
## restorable copy while the directory is still there.
Comment on lines +906 to +909
if [[ $BACKUP_KEEP_DIR -eq 0 ]]; then
rm -rf "$backup_dir"
else
logMessage INFO "Keeping uncompressed backup directory: $backup_dir"
fi

# Clean up directory version (keep archive)
rm -rf "$backup_dir"
else
logMessage ERROR "Failed to create final backup archive"
exit 1
Expand All @@ -850,8 +934,8 @@ case "${BACKUP_COMMAND_PARAMS[0]}" in
;;
list|ls)
echo "Available backups:"
if [[ -d "$(pwd)/.roll/backups" ]]; then
ls -la "$(pwd)/.roll/backups/" | grep -E '^d.*[0-9]{10}$|^-.*backup_.*\.tar'
if [[ -d "${BACKUP_BASE_DIR}" ]]; then
ls -la "${BACKUP_BASE_DIR}/" | grep -E '^d.*[0-9]{10}$|^-.*backup_.*\.tar'
else
echo "No backups found."
fi
Expand All @@ -861,14 +945,14 @@ case "${BACKUP_COMMAND_PARAMS[0]}" in
backup_id="${BACKUP_COMMAND_PARAMS[1]}"

# First check if directory exists (uncompressed backup)
metadata_file="$(pwd)/.roll/backups/$backup_id/metadata/backup.json"
metadata_file="${BACKUP_BASE_DIR}/$backup_id/metadata/backup.json"
if [[ -f "$metadata_file" ]]; then
cat "$metadata_file" | jq '.' 2>/dev/null || cat "$metadata_file"
else
# Look for compressed archive
archive_file=""
for ext in ".tar.gz" ".tar.xz" ".tar.lz4" ".tar"; do
potential_file="$(pwd)/.roll/backups/backup_${ROLL_ENV_NAME}_${backup_id}${ext}"
potential_file="${BACKUP_BASE_DIR}/backup_${ROLL_ENV_NAME}_${backup_id}${ext}"
if [[ -f "$potential_file" ]]; then
archive_file="$potential_file"
break
Expand All @@ -877,7 +961,7 @@ case "${BACKUP_COMMAND_PARAMS[0]}" in

if [[ -z "$archive_file" ]]; then
# Also check for generic archive names
archive_file=$(ls "$(pwd)/.roll/backups"/*"$backup_id"*.tar* 2>/dev/null | head -1)
archive_file=$(ls "${BACKUP_BASE_DIR}"/*"$backup_id"*.tar* 2>/dev/null | head -1)
fi

if [[ -n "$archive_file" ]]; then
Expand Down
17 changes: 16 additions & 1 deletion commands/backup.help
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,19 @@ ROLL_USAGE=$(cat <<EOF
--name=NAME Give backup a custom name
--description=TEXT Add description to backup metadata

\033[33mOutput Options:\033[0m
--output-dir=PATH Write the finished archive to PATH instead of .roll/backups/.
Created if missing and checked for writability before the backup
starts. Volumes are still staged in .roll/backups/ and retention
cleanup never touches PATH, so a shared drop directory holding
other environments' archives is safe.
--archive-name=NAME Basename for the finished archive; the compression extension is
appended (default: backup_<env-name>_<backup-id>). The 'latest'
symlink is only maintained when the archive stays in .roll/backups/.
--keep-dir Keep the uncompressed backup directory in .roll/backups/ as well as
the archive (removed by default). 'roll restore' reads the directory
form, so this is what leaves a restorable backup in the project.

\033[33mDuplication Options:\033[0m
--duplicate-name=NAME Replace ROLL_ENV_NAME in backup for duplication
--duplicate-domain=DOMAIN Replace TRAEFIK_DOMAIN in backup for duplication
Expand All @@ -54,9 +67,11 @@ ROLL_USAGE=$(cat <<EOF
backup list # Show available backups
backup info 1672531200 # Show backup details
backup clean 7 # Remove backups older than 7 days
backup all --output-dir=/srv/builds/latest --archive-name=shop-restore-only
# Drop shop-restore-only.tar.gz in a delivery directory

\033[33mNotes:\033[0m
• Backups are stored in .roll/backups/ directory
• Backups are stored in .roll/backups/ directory unless --output-dir is given
• Service detection is automatic based on environment configuration
• All backups include metadata for restoration tracking
• Use 'roll restore' command to restore from backups
Expand Down
2 changes: 1 addition & 1 deletion version
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.7.0.6
0.7.1
Loading