Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
A single-command bittorrent distribution system, based on Twitter's Murder
Python
Branch: master

Merge pull request #23 from tomas-didziokas/master

moved log_dir creation to main thread
latest commit 02b0ccfd91
@russss authored

README.md

About

Herd is a torrent-based file distribution system based on Murder. It allows for quick and easy transfer of small and large files. You can probably use it for other things too.

Herd requires no extra Python modules and includes everything needed for destinations including its own (lightly modified) copy of BitTornado.

Herd has been updated from its fork Horde which removed limitations around large file transfers, peer seeding, and python integration. Herd now has the same capabilities as Horde in that regard.

Differences from Murder

Murder was Twitter's original BitTorrent-based file distribution system. It's pretty dependent on Capistrano and requires that a separate tracker process is started before you run the deploy task.

Herd spawns its own tracker in the background which makes it really trivial to integrate into whatever deployment system you like.

Requirements

Herd needs Python > 2.5 and argparse. If you're using python 2.7+ nothing else is needed as argparse was added to the standard library.

Argparse, if needed, can be installed with easy_install or pip:

pip install argparse
easy_install argparse

Herd also requires that key based passwordless authentication is enabled on all of your target destinations.

Install

git clone https://github.com/russss/Herd
cd Herd && sudo python setup.py install

Herd also supports pypi_server hosting.

Usage

With a hosts file that includes a list of the hosts you want to copy to:

herd myfile.tar.gz /path/to/destination.tar.gz hosts_file

Using a hosts list that is a single string comma separated:

herd myfile.tar.gz /path/to/destination.tar.gz --hostlist "host1,host2,host3"

Reading the host list from stdin.

cat /tmp/hosts_file | herd myfile.tar.gz /path/to/destination.tar.gz hosts_file

More options:

herd --help

Python Integration

Herd can also be imported as a python module. This makes integration into existing projects easy. One would simply need to:

import herd.herd as herd
herd.run_with_opts('localfile', 'remotefile', hostlist='server1,server2')

Credits

Something went wrong with that request. Please try again.