📡 A protocol to quickly transfer software builds (reference Go implementation)
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
archiver Get rid of atomic.* calls, use mutexes instead. Closes itchio/itch#1897 Jul 9, 2018
bsdiff Closes itchio/itch#1808 May 28, 2018
compressors switch to itchio/go-brotli Jan 9, 2018
counter Switch to protobufs for overlay. Closes itchio/itch#1837 May 29, 2018
crc32c 👕 + 📝 Mar 30, 2016
ctxcopy Introduce ctxcopy, make archiveHealer even more cancellable Apr 14, 2018
decompressors decompressors/cbrotli => decompressors/brotli (it's pure go) May 22, 2018
eos Work around sourceforge's terrible HTTP hygiene Jul 14, 2018
multiread Add lock to avoid race condition in multiread Apr 30, 2018
pools GetReader needs to return a reader at the start of the file, so it ca… May 25, 2018
pwr Introduce RediffSzieLimit Oct 3, 2018
scripts Simplify ci script Apr 16, 2018
splitfunc comments are the salt of your code ~ they're white and grainy? Apr 15, 2016
state consumer: allow all method calls on nil consumer. patcher: use Progre… May 29, 2018
taskgroup Introduce multiread, taskgroup Apr 14, 2018
tlc Introduce 'wrapping', see itchio/itch#2147 Sep 27, 2018
werrors Introduce context in more places for more cancellation Apr 14, 2018
wire Get rid of go-humanize May 22, 2018
wrand Split tests, make them faster, aim for more coverage Dec 31, 2016
wsync Introduce multiread, taskgroup Apr 14, 2018
wtest Closes itchio/itch#1808 May 28, 2018
zstdsource Switch from go-errors/errors to pkg/errors Mar 26, 2018
.gitignore Move around tests a bit, split pwr.StateConsumer into its own package… Oct 5, 2016
.gitlab-ci.yml Update CI config to new platform tags, split into script Mar 1, 2016
LICENSE Initial import Jan 11, 2016
README.md Move bsdiff into its own package, add rsync package's original license Oct 28, 2016

README.md

wharf

build status codecov Go Report Card GoDoc MIT licensed

wharf is a protocol for incrementally transferring software builds over the network using minimal time/bandwidth.

It is used in production at https://itch.io to allow creators to quickly iterate & players to keep their library always up-to-date.

This repository contains the reference golang implementation of the wharf protocol, along with the reference protobuf definition files.

The complete spec is available online, as a book:

And can be contributed to via its GitHub repository:

See also

butler is the https://itch.io command-line tool and is a wharf client. It's the easiest way to try out wharf without having to code anything yourself.

Hacking on wharf

wharf is a pretty typical golang project, all its dependencies are open-source, it even has a few tests.

Regenerating protobuf code

protoc --go_out=. pwr/*.proto

protobuf v3 is required, as we use the 'proto3' syntax.

The tlc (containers) and bsdiff packages work similarly.

License

Licensed under MIT License, see LICENSE for details.

Contains modified code from the following projects: