Backup Benchmarks

Test v2 (Dec 2022)

Goals

Compare Borg 1.2 to Borg 2.0 on older and newer Python versions, as well as Restic.
Compare normal- and high latency connections.
Compare BDP and BBR network optimizations.

Key findings:

Borg 2.0 with Python 3.11 is about 5 to 20% faster on normal- and high latency connections, uses less CPU and about the same memory. The difference was smaller, when the focus was on compression.
CPU time is drastically lower for Borg 2.0. As much as 30%, unless more compression is involved.
Final backup size was the same for Borg 1.2 and 2.0, 10% more for Restic with default settings, when using the same Zstdandard compression.
Restic took 2-4x longer for new backups and took ~2x memory. It's also reading and writing much more data than Borg.
Restic is slightly faster when no data has changed or on a high latency connection. The latter may be due to using 5 parallel upload connections.
Network optimizations to deal with high-bandwidth/high latency connections yielded only small speed improvements (BBR and BDP)
Results were relatively stable across multiple runs of the same scenario.

Benchmark subjects:

"classic" Borg on older Python: Github
"new" Borg on Python 3.11: Github
Restic on Go: Github

Methodology

Backup target: BorgBase server in EU region, off-peak hour
Backup client specs:
- 1 Core, 30GB SSD Storage, 1GB Memory, 1G network
- VPS hosted with OneProvider in Sydney and Nuremberg
Test data: 19.5 GB total, 43921 files
- Corpus part 1:
  - ubuntu-22.10-desktop-amd64.iso, 3.8 GB
  - Visual Genome, images part 2, 5.5 GB
- Corpus part 2:
  - wikidatawiki-20221201-pages-articles-multistream10.xml-p13998368p15498367.bz2, uncompressed, 11 G
Test tasks:
- initial backup of corpus part 1 (create-1)
- subsequent backup with corpus part 2 added (create-2)
- subsequent backup without new data (create-3)
- pruning 50% (prune-1)
Measurements taken by GNU Time:
- Time: wall clock
- CPU usage: user time, system time
- Final backup size: in GB
- Memory usage: max RSS
- File system inputs: Data read in blocks of 512 bytes
- File system outputs: Data written in blocks of 512 bytes

Test Scenarios

normal latency: backup nearby server with connection in Germany, 25ms ping
high latency: backup server on high latency connection in Australia, 270ms ping
high latency with optimized BBR
high latency with optimized BDP

Test machine setup

OS: Debian GNU/Linux 11 (bullseye)

Initial setup steps:

apt update && apt install -y tmux time rsync nload ncdu
ssh-keygen -o -a 100 -t ed25519
echo 'StrictHostKeyChecking accept-new' > ~/.ssh/config

Test commands

Results will be written to results folder.
See tests.sh or results folder for full commands.

source tests.sh

export BORG_REPO=ssh://...
export RESTIC_REPOSITORY=rest:https://...

test_borg-12
test_borg-20
test_restic-14

Feedback and shortcomings

Empty cache between runs to avoid seeing lower Reads due to OS-level caching: sync; echo 3 > /proc/sys/vm/drop_caches (Thanks to Michael from the Restic forum)

Test v1 (Oct 2022)

Test combinations of Python and Borg versions locally using the built-in benchmark command. Uses pyenv to set up different versions and virtualenvs. See here for more.

About BorgBase

BorgBase offers simple and secure backup storage for open source backup tools, like Borg and Restic. Manage all your backup repositories in one place via web interface or API. Monitor usage, quotas and security settings, and choose to be alerted about stale backups. Every repo is isolated, encrypted and optionally append-only to protect against ransomware attacks.

borgbase/benchmarks

ErrorLooks like something went wrong!

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Languages

Error
Looks like something went wrong!