Centralized Docker backup and update management CLI application for managing multiple Docker hosts.
Repository: https://github.com/DeviantEng/docker-manager
- β Automated Backups - Stop, backup, restart projects safely (preserves container state)
- β Flexible Scheduling - Daily, weekly, biweekly, monthly per project
- β Smart Updates - Automatic updates after backups (configurable)
- β Retention Management - Keep last N backups per project with auto-cleanup
- β Selective Backups - Exclude volumes, patterns, or backup only compose files
- β Multi-Host - Manage multiple Docker hosts from one central machine
- β Notifications - ntfy integration with retry logic for reliable alerts
- β SSH-based - Secure remote execution via SSH keys
- β YAML Configuration - Easy, readable configuration with per-project overrides
- β State Preservation - Containers return to their original state (running/stopped)
- β Docker Prune - Weekly cleanup of unused images, containers, build cache on hosts
- β Log Cleanup - Automatic removal of old log files (default: 30 days retention)
- Python 3.8+
- SSH access to Docker hosts (root with SSH keys configured)
- NFS share mounted on all hosts and admin machine
pigzinstalled on Docker hosts (optional, for faster compression)
# Clone repository
git clone https://github.com/DeviantEng/docker-manager.git /opt/docker-manager
cd /opt/docker-manager
# Create virtual environment
python3 -m venv venv
source venv/bin/activate
# Install dependencies
pip install -r requirements.txt
# Configure
cp docker-manager.yml.example docker-manager.yml
nano docker-manager.yml
# Create log directory
mkdir -p /var/log/docker-manager
# Test
./docker-manager.py test-ssh
./docker-manager.py test-notify
./docker-manager.py list
# Run first backup
./docker-manager.py backup --host docker01 your-project
# Setup daily cron
crontab -e
# Add: 0 2 * * * /opt/docker-manager/venv/bin/python3 /opt/docker-manager/docker-manager.py runSee SETUP.md for detailed installation instructions.
global:
hosts:
docker01:
ip: 192.168.1.101
docker_root: /opt/docker
backup:
root: /mnt/nfs/docker-backups
default_schedule: weekly
default_retention: 4
notifications:
enabled: true
provider: ntfy
ntfy:
server: https://ntfy.example.com
topic: docker-manager
username: user
password: passprojects:
# Critical service - daily backups, keep 14
vaultwarden:
retention: 14
schedule: daily
# Skip backup, only update
musicbrainz:
behavior: update_only
# Exclude cache/logs from backup
plex:
schedule: weekly
exclude_patterns:
- "*/Logs/*"
- "*/Cache/*"global:
docker_prune:
enabled: true
schedule: weekly # daily, weekly, biweekly, monthly
include_volume_prune: true
# marker_file: /path/to/.last-docker-prune # optional; default: {backup.root}/.last-docker-prune
log_retention_days: 30 # Remove log files older than N daysPer-host overrides: Add docker_prune under a host to disable (enabled: false), change schedule, or skip volume prune.
See docker-manager.yml.example for all options.
# Scheduled via cron (respects schedules, performs cleanup)
./docker-manager.py run# Backup operations
./docker-manager.py backup all # All projects
./docker-manager.py backup --host docker01 # All on one host
./docker-manager.py backup --host docker01 vaultwarden # Specific project
# Update operations
./docker-manager.py update all # Check all for updates
./docker-manager.py update vaultwarden # Update specific project
# Combined (backup then update)
./docker-manager.py run --force # Force all, ignore schedules
./docker-manager.py run --host docker01 vaultwarden # One project
# Maintenance
./docker-manager.py cleanup # Remove old backups and logs per retention
./docker-manager.py docker-prune # Run docker prune on hosts (manual, bypasses schedule)
./docker-manager.py docker-prune --host docker01 # Prune specific host only
./docker-manager.py list # Show all discovered projects
./docker-manager.py test-ssh # Test connectivity
./docker-manager.py test-notify # Test notifications- Check if backup is due based on schedule
- SSH to Docker host
- Check container state (running/stopped)
- Stop containers (if running)
- Create compressed tar backup (excluding configured patterns)
- Check for updates (if
backup_then_update) - Start containers (if they were running)
- Clean up old backups per retention policy
The schedule setting controls both backup AND update frequency:
daily- Backup + update every dayweekly- Backup + update once per week (if last backup >7 days)biweekly- Backup + update every 2 weeksmonthly- Backup + update once per month
Schedules are checked by parsing timestamps from backup filenames.
backup_then_update- Backup, then check for updates (default)backup_only- Backup without checking for updatesupdate_only- Check for updates without backing up
Containers are always returned to their original state:
- If running before backup β Stopped β Backed up β Updated β Started
- If stopped before backup β Backed up β Remain stopped (no update check)
When run is executed, docker prune runs on each host if due (same schedule logic as backups). It removes:
- Stopped containers
- Unused images
- Build cache
- Orphaned volumes (if
include_volume_prune: true)
Schedule is tracked per-host via a marker file (default: {backup.root}/.last-docker-prune).
After each run, log files older than log_retention_days (default 30) are removed from log_dir.
Backups are named: {hostname}-{project}-{timestamp}.tar.gz
Examples:
docker01-vaultwarden-20241204-103000.tar.gz
docker01-jellyfin-20241204-103005.tar.gz
docker02-plex-20241204-103010.tar.gz
Each backup contains:
- docker-compose.yml and related files
- All volumes (unless excluded)
- .env files
- Project directory structure
Control what gets backed up:
global:
backup:
default_exclude_patterns:
- "*.log"
- "*.sock"
- "*/cache/*"
- "*/logs/*"projects:
plex:
exclude_patterns:
- "*/Logs/*"
- "*/Cache/*"
- "*/Crash Reports/*"See EXCLUSIONS.md for pattern examples and guidance.
Notifications are sent via ntfy with automatic retry logic.
Backup Summary:
π³ Docker Manager: 12 backups completed
πΎ Backup Complete
ββββββββββββββββββββ
Projects: 12 (45 containers)
β
Successful: 12
Total Size: 5.2GB
Update Summary:
π³ Docker Manager: 3 updates applied
π Updates Complete
ββββββββββββββββββββ
Checked: 12 projects
β
Updated: 3
βοΈ Up-to-date: 9
To restore a backup:
# 1. SSH to host
ssh root@docker01
# 2. Stop service
cd /opt/docker/vaultwarden
docker compose down
# 3. Extract backup
cd /opt/docker
tar -xzf /mnt/nfs/docker-backups/docker01-vaultwarden-20241204-103000.tar.gz -C vaultwarden/
# 4. Start service
cd vaultwarden
docker compose up -d# Check logs
tail -f /var/log/docker-manager/docker-manager-$(date +%Y%m%d).log
# Review last run
tail -100 /var/log/docker-manager/docker-manager-$(date +%Y%m%d).log
# Check backup sizes
du -sh /mnt/nfs/docker-backups/*
# List backups per project
ls -lh /mnt/nfs/docker-backups/ | grep vaultwardensource venv/bin/activate
pip install -r requirements.txt# Test manually
ssh root@192.168.1.101 "echo OK"
# Check SSH keys
ls -la ~/.ssh/
# Use built-in test
./docker-manager.py test-ssh# Test notification
./docker-manager.py test-notify
# Check configuration
grep -A 6 "notifications:" docker-manager.yml# Check schedule (might not be due yet)
./docker-manager.py run --force
# Check logs
tail -100 /var/log/docker-manager/docker-manager-$(date +%Y%m%d).log/opt/docker-manager/
βββ docker-manager.py # Main application
βββ docker-manager.yml # Your configuration (gitignored)
βββ docker-manager.yml.example # Example configuration
βββ requirements.txt # Python dependencies
βββ .gitignore # Git ignore rules
βββ README.md # This file
βββ SETUP.md # Detailed setup guide
βββ EXCLUSIONS.md # Exclusion pattern guide
βββ venv/ # Virtual environment (gitignored)
/var/log/docker-manager/
βββ docker-manager-YYYYMMDD.log # Daily logs
- Protect
docker-manager.ymlwithchmod 600(contains credentials) - Use SSH keys (not passwords) for host access
- Secure your NFS share with proper permissions
- Review logs for sensitive information before sharing
Contributions welcome! Please:
- Fork the repository
- Create a feature branch
- Make your changes
- Test thoroughly
- Submit a pull request
MIT
- Issues: https://github.com/DeviantEng/docker-manager/issues
- Logs: Check
/var/log/docker-manager/for detailed information
DeviantEng - https://github.com/DeviantEng
- β Automated Backups - Stop, backup, restart projects safely
- β Flexible Scheduling - Daily, weekly, biweekly, monthly per project
- β Smart Updates - Automatic updates after backups (configurable)
- β Retention Management - Keep last N backups per project
- β Selective Backups - Exclude volumes, backup only compose files
- β Multi-Host - Manage multiple Docker hosts from one place
- β Notifications - ntfy integration for status updates
- β SSH-based - Secure remote execution via SSH
- β YAML Configuration - Easy, readable configuration
- Python 3.8+
- SSH access to Docker hosts (root with SSH keys)
- NFS share mounted on all hosts and admin machine
pigzinstalled on Docker hosts (optional, for faster compression)
# Clone repository
git clone https://github.com/yourusername/docker-manager.git /opt/docker-manager
cd /opt/docker-manager
# Install Python dependencies
pip3 install -r requirements.txt
# Make executable
chmod +x docker-manager.py
# Create symlink (optional, for convenience)
ln -s /opt/docker-manager/docker-manager.py /usr/local/bin/docker-manager
# Create log directory
mkdir -p /var/log/docker-managerCopy the example configuration and customize:
cp docker-manager.yml.example docker-manager.yml
nano docker-manager.ymlRequired settings:
global.hosts- Your Docker hosts with friendly names and IPsglobal.backup.root- Path to NFS backup directoryglobal.notifications.ntfy- Your ntfy server credentials
Optional:
projects- Per-project overrides (schedule, retention, exclusions)
See docker-manager.yml.example for detailed configuration options.
# Test SSH connectivity to all hosts
./docker-manager.py test-ssh
# Test notifications
./docker-manager.py test-notify
# List all discovered projects
./docker-manager.py list
# Test backup a single project
./docker-manager.py backup --host docker01 vaultwarden
# Check logs
tail -f /var/log/docker-manager/docker-manager-$(date +%Y%m%d).log# Run scheduled backups and updates (respects schedule config)
docker-manager.py run
# Force run everything regardless of schedule
docker-manager.py run --force
# Run specific host or project
docker-manager.py run --host docker01
docker-manager.py run vaultwarden# Backup all projects
docker-manager.py backup all
# Backup specific project on all hosts
docker-manager.py backup vaultwarden
# Backup all projects on specific host
docker-manager.py backup --host docker01
# Backup specific project on specific host
docker-manager.py backup --host docker01 vaultwarden# Update all projects
docker-manager.py update all
# Update specific project
docker-manager.py update vaultwarden
# Update all on specific host
docker-manager.py update --host docker02
# Update specific project on specific host
docker-manager.py update --host docker02 jellyfin# Clean up old backups (respects retention policy)
docker-manager.py cleanup
# List all projects
docker-manager.py list# Test SSH connectivity
docker-manager.py test-ssh
# Test notifications
docker-manager.py test-notifySet up a daily cron job:
crontab -eAdd:
0 2 * * * /opt/docker-manager/docker-manager.py run
This runs daily at 2 AM, checking each project's schedule and backing up/updating as configured.
global:
# Host definitions
hosts:
docker01:
ip: 192.168.1.101
docker_root: /opt/docker
docker02:
ip: 192.168.1.102
docker_root: /opt/docker
# Backup settings
backup:
root: /mnt/media/nfs/docker-backups
compression: pigz # pigz (faster) or gzip
compression_level: 6 # 1-9
default_retention: 4 # Keep last N backups per project
default_schedule: daily # daily, weekly, biweekly, monthly
# Global exclusion patterns (optional)
default_exclude_patterns:
- "*.log"
- "*.sock"
- "*/cache/*"
- "*/logs/*"
# Update settings
update:
default_behavior: backup_then_update # backup_then_update, backup_only, update_only
# Notifications
notifications:
enabled: true
provider: ntfy
ntfy:
server: https://ntfy.example.com
topic: your-topic
username: your-user
password: your-passProjects not listed use global defaults. Add per-project customization:
projects:
# High-value service: daily backups, keep 10
vaultwarden:
retention: 10
schedule: daily
behavior: backup_then_update
# Skip backup entirely, only update
musicbrainz:
behavior: update_only
schedule: weekly
# Media server: weekly backups, exclude cache
jellyfin:
retention: 6
schedule: weekly
exclude_volumes:
- cache
- transcodes
# Database: daily backup, never auto-update
postgres:
retention: 10
schedule: daily
behavior: backup_only
# Only backup compose files, no volumes
nginx:
retention: 4
exclude_volumes:
- ALL # Special keywordretention- Keep last N backups (overrides default)schedule-daily,weekly,biweekly,monthlybehavior-backup_then_update,backup_only,update_onlybackup_compose-falseto skip compose files (default:true)exclude_volumes- List of volume directories to skip- Use
ALLto exclude all volumes (backup only compose files)
- Use
exclude_patterns- List of file/path patterns to exclude (per-project)- Patterns are passed to
tar --exclude - Merged with
default_exclude_patternsfrom global config - Examples:
"*/Logs/*","*.tmp","*/Cache/*"
- Patterns are passed to
projects:
plex:
retention: 4
schedule: weekly
exclude_patterns:
- "*/Logs/*" # Skip Plex logs
- "*/Cache/*" # Skip Plex cache
- "*/Crash Reports/*" # Skip crash reports
# Metadata is backed up (not excluded)For each project:
- Check if backup is due based on schedule and last backup timestamp
- SSH to Docker host
docker compose down(stop containers)- Create compressed tar backup (excluding configured volumes)
docker compose up -d(restart containers)- Clean up old backups based on retention policy
For projects where behavior allows:
- SSH to Docker host
- Get current image digests
docker compose pull- Compare new vs old digests
- If changed:
docker compose up -d(recreate containers)
Backup schedules are enforced by parsing the timestamp from existing backup filenames:
daily- Backup if last backup was >24 hours agoweekly- Backup if last backup was >7 days agobiweekly- Backup if last backup was >14 days agomonthly- Backup if last backup was >30 days ago
Projects without existing backups are always backed up.
Backups are named: {hostname}-{project}-{timestamp}.tar.gz
Examples:
docker01-vaultwarden-20241204-103000.tar.gz
docker01-jellyfin-20241204-103005.tar.gz
docker02-plex-20241204-103010.tar.gz
To restore a backup:
# 1. SSH to the host
ssh root@docker01
# 2. Stop the service
cd /opt/docker/vaultwarden
docker compose down
# 3. Extract backup over existing directory
cd /opt/docker
tar -xzf /mnt/media/nfs/docker-backups/docker01-vaultwarden-20241204-103000.tar.gz -C vaultwarden/
# 4. Restart service
cd vaultwarden
docker compose up -dπ³ Docker Manager: 12 backups completed
ββββββββββββββββββββ
Total: 14 projects
β
Successful: 12
β Failed: 2
Total Size: 3.2GB
π³ Docker Manager: 3 updates applied
ββββββββββββββββββββ
Checked: 14 projects
β
Updated: 3
βοΈ Up-to-date: 11
π³ Docker Manager: Cleanup Complete
ββββββββββββββββββββ
Backups removed: 5
Space freed: 12.3GB
# Install from requirements.txt
pip3 install -r requirements.txt
# Or install individually
pip3 install paramiko pyyaml requests python-dateutil# Test SSH manually
ssh root@172.16.100.200 "echo OK"
# Check SSH keys exist
ls -la ~/.ssh/
# Use test-ssh command
./docker-manager.py test-ssh# Check logs
tail -100 /var/log/docker-manager/docker-manager-$(date +%Y%m%d).log
# Force run to ignore schedules
./docker-manager.py run --force
# Test specific project
./docker-manager.py backup --host docker01 vaultwarden# Test notification
./docker-manager.py test-notify
# Check configuration
grep -A 6 "notifications:" docker-manager.yml/opt/docker-manager/
βββ docker-manager.py # Main application
βββ docker-manager.yml # Your configuration
βββ docker-manager.yml.example # Example configuration
βββ requirements.txt # Python dependencies
βββ README.md # This file
/var/log/docker-manager/
βββ docker-manager-YYYYMMDD.log # Daily logs
The code is structured for easy extension:
class DockerManager:
def backup_project() # Backup logic
def update_project() # Update logic
def run() # Main orchestration
class Notifier:
def send() # Send notifications# Test SSH
python3 docker-manager.py test-ssh
# Test notifications
python3 docker-manager.py test-notify
# Dry run (view discovered projects)
python3 docker-manager.py list- SSH keys should be properly secured (not password-based auth)
- Protect
docker-manager.ymlas it contains ntfy credentials:chmod 600 docker-manager.yml - Backups contain all project data including secrets - secure the NFS share
- Logs may contain sensitive information - review log permissions
Contributions welcome! Please:
- Fork the repository
- Create a feature branch
- Make your changes
- Submit a pull request
MIT
For issues, check logs at /var/log/docker-manager/ or open an issue on GitHub.
Edit /opt/docker-manager/docker-manager.yml:
global:
hosts:
docker01:
ip: 172.16.100.200
docker_root: /opt/docker
backup:
root: /mnt/media/nfs/docker-backups
default_retention: 4
default_schedule: daily
notifications:
enabled: true
provider: ntfy
ntfy:
server: https://ntfy.example.com
topic: your-topic
username: user
password: pass
projects:
vaultwarden:
retention: 8
schedule: daily
jellyfin:
exclude_volumes:
- cache
- transcodeshosts- Define Docker hosts with friendly namesbackup.root- NFS path for backupsbackup.compression-pigz(faster) orgzipbackup.default_retention- Keep last N backupsbackup.default_schedule-daily,weekly,biweekly,monthlyupdate.default_behavior-backup_then_update,backup_only,update_only
Projects not listed use global defaults. Override per project:
retention- Keep last N backups for this projectschedule- Backup frequency for this projectbehavior- Override update behaviorbackup_compose- Set tofalseto skip compose filesexclude_volumes- List of volume directories to skip- Use
ALLto exclude all volumes (backup only compose)
- Use
# Test SSH connectivity
docker-manager test-ssh
# Test notifications
docker-manager test-notify
# List all discovered projects
docker-manager list# Backup all projects (respects schedule)
docker-manager run
# Force backup all (ignore schedule)
docker-manager run --force
# Backup specific project on all hosts
docker-manager backup vaultwarden
# Backup all projects on specific host
docker-manager backup --host docker01
# Backup specific project on specific host
docker-manager backup --host docker01 vaultwarden# Update all projects (respects behavior settings)
docker-manager update all
# Update specific project
docker-manager update vaultwarden
# Update all on specific host
docker-manager update --host docker02# Clean up old backups (respects retention)
docker-manager cleanup
# Show status (not yet implemented)
docker-manager statusRun daily at 2 AM:
crontab -eAdd:
0 2 * * * /usr/local/bin/docker-manager run
For each project:
- Check if backup is due based on schedule
- SSH to host
docker compose down(stop containers)- Create
tar.gzbackup (with exclusions) docker compose up -d(restart containers)- Enforce retention policy (delete old backups)
For each project (if behavior allows):
- SSH to host
- Get current image digests
docker compose pull(check for updates)- Compare new vs old digests
- If changed:
docker compose up -d(recreate containers)
When running docker-manager run:
- Parse last backup timestamp from filename
- Calculate days since last backup
- Compare to project schedule setting
- Backup if due, skip if not
Example:
- Project with
dailyschedule, last backup yesterday β Skip - Project with
weeklyschedule, last backup 8 days ago β Backup - Project with no backups β Always backup
{hostname}-{project}-{timestamp}.tar.gz
Examples:
docker01-vaultwarden-20241204-103000.tar.gz
docker01-jellyfin-20241204-103005.tar.gz
docker02-plex-20241204-103010.tar.gz
π³ Docker Manager: 12 backups completed
ββββββββββββββββββββ
Total: 14 projects
β
Successful: 12
β Failed: 2
Total Size: 3.2GB
π³ Docker Manager: 3 updates applied
ββββββββββββββββββββ
Checked: 14 projects
β
Updated: 3
βοΈ Up-to-date: 11
π³ Docker Manager: Cleanup Complete
ββββββββββββββββββββ
Backups removed: 5
Space freed: 12.3GB
# Test SSH manually
ssh root@172.16.100.200 "echo OK"
# Check SSH keys
ls -la ~/.ssh/# Reinstall dependencies
pip3 install -r /opt/docker-manager/requirements.txt# Check logs
tail -100 /var/log/docker-manager/docker-manager-$(date +%Y%m%d).log
# Run with verbose output
docker-manager run --force# Test notification
docker-manager test-notify
# Check config
cat /opt/docker-manager/docker-manager.yml | grep -A 6 notificationsprojects:
postgres-db:
retention: 10
schedule: daily
behavior: backup_only # Don't auto-updateprojects:
jellyfin:
retention: 4
schedule: weekly
exclude_volumes:
- cache
- transcodesprojects:
documentation:
retention: 2
schedule: monthly
exclude_volumes:
- ALLprojects:
musicbrainz:
behavior: update_only
schedule: weekly- Uses root SSH keys for host access
- Stores ntfy credentials in config file (protect with
chmod 600) - Backups contain all project data including secrets
- Ensure NFS share has proper permissions
- Python 3.8+
- SSH access to Docker hosts (root with keys)
- NFS share mounted on all hosts and admin machine
pigzinstalled on Docker hosts (optional, for faster compression)
/opt/docker-manager/
βββ docker-manager.py # Main application
βββ docker-manager.yml # Configuration
βββ requirements.txt # Python dependencies
βββ README.md # This file
/var/log/docker-manager/
βββ docker-manager-YYYYMMDD.log # Daily logs
MIT
Check logs at /var/log/docker-manager/ for detailed information.