Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Using Borg on (device-managed) SMR drives #2252
~FAQ entry could look like this
(Removed, see below
Test was conducted using a Seagate Archive v2 8 TB (ST8000AS0002) which is pretty much the only model available to consumers. Kernel is Linux 4.9.13.
With an ext4 formatted like above and current master branch ~165 MB/s write speed are achieved when processing large files. The drive works continuously, seeking rarely, so apparently Borg and ext4 work nicely here together such that data is written directly to their zones and not first to the buffer and then later to the final zone.
About once every ~30 minutes either ext4/jbd or the drive, not sure who, decides to do something resulting in a short IO stall where the drive performs a lot of seeking for <10 seconds with immediate return to normal afterwards. I assume that this is related to ext4's journal.
When the drive remains idle for some time (5-10 mins) it seems to flush the internal buffer. In this case it seems to be done after less than a minute.
Read performance in "borg check" is good (130-195 MB/s, about ~140 MB/s average).
Average repo write speed is 93 MB/s or 132 MB/s if you compensate for deduplication.
1.0 (5 MB segments): first ~14 GB okay-ish (40 MB/s), then starts to stall, jbd (ext4 journal) and borg iowait increases a lot (90-99 %), disk starts to seek a lot. Average write speed is erratic and varies in the <10 MB/s area. After aborting + umounting FS and some idle time drive goes, acoustically speaking, nuts. Seems to flush it's buffer again. Edit: Took over an hour before the drive was done.
1.1 on vanilla ext4: FS creation, mount, umount take some time. Performance seems to be almost identical to options given above (which does not mean that they don't make a difference - just not for Borg. This could be caused by Borg already being well-matched to the characteristics of these drives, so Borg doesn't need special FS options).
Here's a graph (made using JSON output + with timestamps + matplotlib + numpy gradient):
(Note: different settings + source as in other tests, not directly comparable)
Blue = instantaneous numerical differentiation of transfer amount, therefore noisy as expected, orange = running linear average (nominal 1 minute window) over that, red = latency
Notice how the latency regularly spikes, sometimes to very large levels (around minute 80) and for longer times.
I'm not sure why it's a bit faster with less spread over the first ~240 GB.
The intervals where transfer rate = 0 but latency = no spike is where duplicates where in the source, nothing unusual.
latency := running difference between progress updates from borg minus the nominal rate, so anything that blocks borg will show up here - but there shouldn't be anything except the SMR drive causing it (source is a regular, idle drive, no network).
Edit: Fancier graph: