Skip to content
Maintain disk images or other large files indefinitely on the Sia network.
Python Makefile
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
tests
.gitignore
LICENSE.txt
README.md Change the default block size to 80MiB per the issue #2 discussion. Nov 13, 2019
setup.py
siaslice.py

README.md

Sia Slice

...is a small program that maintains a mirror of a large file on the Sia decentralized storage network. The envisioned use case is storing a long-term, low-cost disk image, with periodic updates, for backups and archives on The Cloud™.

The basic idea behind Sia Slice is to chop up a single block device (or other large, monolithic file) into thousands of 80MiB chunks, LZMA-compress those chunks, and then upload them to Sia. That way, the next time you sync, you only have to upload the chunks that have changed since the last sync. Sync operations always construct a complete and identical mirror; there is no history, and there are no "full" or "incremental" snapshots. This minimizes both the complexity of the program and the storage requirements on Sia.

Besides, if you need those features, you can simply use any filesystem you choose on top of the original device or disk image. This ability is what makes Sia Slice unique: In contrast to other synchronization programs like Siasync and Repertory that operate at the file level, Sia Sice operates at the block level.

Curses screenshot

Sia screenshot

Sia Slice was written for GNU/Linux systems with Python 3.7 or later. Ports to other platforms should be possible with minimal effort.

The author uses Sia Slice weekly to mirror his Btrfs backup drive.

Installation

pip install https://github.com/YoRyan/sia-slice

Usage

You must provide the API password required to communicate with Sia, either through the command line or by setting $SIA_API_PASSWORD. I recommend using the environment variable:

export SIA_API_PASSWORD=xxxxxx

To copy the contents of /dev/sdb1 to a new Sia folder at /backupdrive:

siaslice --mirror /dev/sdb1 backupdrive

(To sync again, just run the same command. Sia Slice will locate the previous uploads and determine which blocks need to be re-uploaded.)

To download all Sia Slice data from /backupdrive and reassemble it on /dev/sdc1:

siaslice --download /dev/sdc1 backupdrive

Finally, Sia Slice writes a timestamped state file that can be used to resume any mirror or download operation in the event of a program crash, network interruption, or other catastrophic event.

To resume a stalled mirror operation:

siaslice --resume siaslice-mirror-20191024-1522.dat

Notes

Sia is an emerging technology, and I jumped through a lot of hoops to write this software. For the curious, I have written a companion blog post.

For contributors and forkers, some tests are located in tests/.

pip install -e .
pip install asynctest
cd tests/
python -m unittest ...
You can’t perform that action at this time.