Massively scalable, cloud-backed distributed block device for Linux and VMs
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Welcome to

This repository contains the (formerly oneroot) distributed block device, whose aim is to eventually become the 'git for storage'. It is currently mostly useful for local use, and for creating locally writable mounts that can be exported simply by placing the chunked files it creates on an HTTP somewhere, for instance on S3 or Google Storage. It supports fetching remote objects on demand, but currently you have to push them to the cloud or your own HTTP server, e.g, using a tool like rclone or rsync. The block devices operates over a compressed and encrypted database-like structure, somewhat similar to LevelDB or RocksDB, but optimized for block storage. The code was forked from Bromium's internal ".swap" storage engine, which is the storage component of that company's ground-breaking micro-virtualization security technology, and has (anecdotally) been used to launch more VMs than Amazon AWS. Today, development is sponsored by, and the main developer is Jacob Gorm Hansen, who also led the work at Bromium.


To build on debian/ubuntu, you need to install:

  • cmake
  • ninja-build
  • liblz4-dev
  • libcurl4-gnutls-dev (or other libcurl-dev package)
  • uuid-dev
  • libssl-dev

And to run well you should have an entropy gathering service like rng-tools or haveged installed and running.

To run with the example scripts you also need to install:

  • xfsprogs

Then you can build with:


All that make does here is act as a think wrapper around the usual cmake and ninja dance, so feel free to use that instead if you prefer.


You can run mindcastle's NBD backend with (must be root, prefix with sudo if that is your thing):

modprobe nbd
oneroot mydisk.swap ./

oneroot implements a user-space block device on top of Linux' nbd module. The mydisk.swap file is a tiny meta-data text file that will be created if not there already. The actual disk contents will go inside a directory called swapdata-UUID, where UUID gets randomly generated. As long as oneroot is running, you now have an empty block device that you can format and use as any other block device. (To not have to do this manually each time, the script "" takes care to do this automically, you can edit the script to choose the type of filesystem or perform automated tasks on, e.g, automatically syncing files to the file system and then unmounting it when done.)

If you peek inside swapdata-UUID, you will see a lot of regularly-sized files (currently the chunking size is set to 1MiB) with some insanely long file names. The file names are derived from the sha512 hashes of the file contents, as you can verify by running the 'sha512' tool against one of them (the hashes get truncated at 256 bits when used for filenames). If you wanted to publish it on an HTTP server, say running out of /srv/http, you would do something line this (assuming the HTTP server serves files out /srv/http and there is a http user on the system that the server is running as):

cp -r swapdata-UUID /srv/http
chown -R http /srv/http/swapdata-UUID 

Then, to be able to mount HTTP-repo from another host, you would copy and edit mydisk.swap to look something like this:



Here, we added the fallback= line to point to the location of the HTTP server acting as the storage backend.

Then, on the client machine, you could connect the block device with:

oneroot mydisk.swap ./

Where "./" is the path to the default stagechange script, which will format the volume if necessary, and mount and later unmount it under /tmp/mnt-UUID. The script interacts with the oneroot process using signals, and the user can do this too. For example, to trigger a clean unmount and shutdown, just do a

kill -1 PID

Where PID is that of the first oneroot process, you should see this logged by the script on startup. See the script comments for more details.