PersonalBackupPipeline
title: Personal Data Backup Pipeline type: technique created: 2026-05-21 last_updated: 2026-05-21 related: ["radar/techniques/DockerSecuritySelfHosting", "radar/techniques/PackerProxmoxTemplates"] sources: ["https://justinpaulin.com/2023/11/27/my-personal-data-backup-pipeline/"] radar_quadrant: Techniques radar_ring: Assess radar_position: inner
An automated, code-defined backup strategy for self-hosted data applying the 3-2-1 rule via rclone, with scheduled sync jobs pushing to multiple cloud destinations.
The 3-2-1 backup strategy is the baseline for data durability:
- 3 copies of data total
- 2 on different storage media
- 1 offsite (geographically separate)
For self-hosted environments: the live data is copy one, a local NAS or external drive is copy two (different media), and a cloud provider is copy three (offsite).
rclone is a CLI tool for syncing files to and from cloud storage (S3, Backblaze B2, Google Drive, Dropbox, and 50+ others). It supports encryption, bandwidth throttling, and delta sync — only transferring changed files.
Key rclone features for backup pipelines:
-
rclone sync— mirrors source to destination, deleting destination files not in source. -
rclone copy— copies new/changed files without deleting destination files. -
rclone crypt— transparent encryption remote; wraps another remote with client-side AES-256 encryption before upload. -
rclone bisync— bidirectional sync for two-way replication.
A typical automated pipeline:
- Local backup job runs nightly via cron or systemd timer.
- rclone copies changed files from the live data directory to a local NAS (copy two).
- rclone syncs from the live directory to an encrypted cloud remote (copy three).
- A separate job verifies backup integrity with
rclone check. - Alerts fire (via ntfy or email) on job failure.
For Docker-hosted services, volumes are backed up by stopping the container, snapshotting the volume directory, and restarting — ensuring backup consistency.
| Provider | Cost model | Notes |
|---|---|---|
| Backblaze B2 | $6/TB/month | S3-compatible, low egress cost |
| AWS S3 Glacier | $1/TB/month | Retrieval latency; suitable for archives |
| Cloudflare R2 | $15/TB/month storage, $0 egress | No egress fees; good for frequent restores |
Personal Data Backup Pipeline sits in the Assess ring of the Techniques quadrant, at inner position. First studied via Justin Paulin's blog (2023-11-27). rclone is a mature, widely-used tool; the pipeline pattern is well-established for self-hosted environments. Inner position reflects immediate applicability to any self-hosted setup with data durability requirements and a low barrier to trial — rclone installs as a single binary, and a working backup job can be configured in under an hour. Remaining gate is a completed automated pipeline with verified restores for at least one active self-hosted service.
