Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
fs
 
 
 
 
lib
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

README.md

rio

Repeatable I/O.

Rio is tooling for repeatable, reproducible, filesystem I/O and transport. It's both a command line tool, and usable as a series of libraries.

Rio is part of the Timeless Stack -- it handles packing, unpacking, syncing, copying, and hashing of Filesets.

You may also want to check out Repeatr, which is the container executor in the Timeless Stack (it uses Rio under the hood to provide snapshotting and decentralized sync of container filesystem images); and Reach, which provides pipelining tooling for managing both Rio and Repeatr with less copy-pasting of hashes.

What?

example of packing

Okay, by example:

rio pack tar /tmp/something/ --target=file://output.tgz

That just created a tar pack of the files you aimed it at.

It also gave you a hash on stdout. That's a content-addressable ID of the pack you just created.

example of unpacking

You can use that ID later to unpack things:

rio unpack tar:879UrF8j7E[...]udF57KpF8 \
   /tmp/unpackhere/ --source=file://output.tgz

It's long. That's because it's a cryptographic hash.

Why is this neat?

You can put that packed data on any server, anywhere, and fetch it again by hash. The hash makes it immutable, and reproducible, even if you don't control the storage.

example of packing to remote storage and using content-addressing

rio pack tar /tmp/something/ --target=ca+https://mybucket.s3.amazonaws.com/assets/

Same drill except... we just:

  • uploaded to a cloud storage host
  • without a specific name -- the hash is used to organize storage (note the "ca+" segment of the url).
  • and still got the same hash.

example of unpacking from remote storage while using content-addressing

Now, hand that hash to someone else:

rio unpack tar:879UrF8j7E[...]udF57KpF8 \
   /tmp/unpackhere/ --source=ca+https:///mybucket.s3.amazonaws.com/assets/

They get your files. Boom. Huge amounts of data. Just one handle to copypaste: that hash.

Building

Rio is built in Golang and uses git submodules to track libraries by hash.

To build Rio, first get the submodules, then set up GOPATH, then use go:

git submodule update --init
GOPATH=$PWD/.gopath go install ./...

You may find the gof script makes this more convenient.

About

Part of the Timeless Stack -- Reproducible I/O. Snapshots of files, transport, content-addressing, wow.

Resources

License

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  
You can’t perform that action at this time.