Skip to content
This repository

Getting Started


  • One or more x86-64 machines.
  • Linux kernel 2.6.27 or later
  • glibc 2.9 or later
  • The corosync and corosync lib package.
  • QEMU 0.13 or later

Compile-time dependencies

  • GNU Autotools
  • pkg-config
  • corosync devel package
  • git (when compiling from source repo)
  • liburcu
  • optional:fuse-devel (for sheepfs)


Compile or install the Corosync packages

Nearly every modern Linux distribution has x86_64 corosync binaries pre-built available via their repositories. We recommend you use these packages if they are available on your distribution.

For debian package based systems:

$ sudo aptitude install corosync libcorosync-dev

For RPM package based systems:

$ sudo yum install corosynclib-devel

For EL6 (RHEL, CentOS, SL, etc), the provided version of corosync is too old and you must install corosync from source. The following command will remove any existing copies of corosync.

$ sudo yum remove corosync corosynclib corosynclib-devel 

If your distribution doesn't provide the corosync packages, or you prefer to compile from source:

$ git clone git://
$ cd corosync
$ git checkout -b flatiron origin/flatiron
$ ./
$ ./configure --enable-nss
$ sudo make install

See also: Corosync Config

Download, build and install QEMU with Sheepdog support

QEMU 0.13 provides built-in support for sheepdog devices. Some distributions provide pre-built versions of this newer version of QEMU. If your distribution has an older version of QEMU or you prefer to compile from source, retrieve the latest QEMU and compile:

$ git clone git://
$ cd qemu
$ ./configure
$ sudo make install
$ cd ..

Download, build and install the Sheepdog server and command line tools

If you want to use a stable sheepdog, please download our source archive of stable branch and build it. Stable branches of each version are listed here. Below is an example of v0.7.6.

$ wget -O sheepdog-v0.7.6.tar.gz
$ tar xvzf sheepdog-v0.7.6.tar.gz
$ cd sheepdog-v0.7.6
$ ./
$ ./configure
$ sudo make install
$ cd ..

If you want to use every feature of the latest sheepdog, please clone our git repository.

$ git clone git://
$ cd sheepdog
$ ./
$ ./configure
$ sudo make install
$ cd ..

Please note, sheepdog supports a "make rpm" target which will generate an rpm package that can be installed on the local machine. To use this installation method, use the following instructions:

$ git clone git://
$ cd sheepdog
$ ./
$ ./configure
$ make rpm
$ sudo rpm -ivh x86_64/sheepdog-0.*
$ cd ..


Setup Sheepdog

  1. To launch the sheepdog daemon, either start it using '/etc/init.d/sheepdog start' or launch the sheepdog daemon by directly calling 'sheepdog ' on each machine of the cluster. Make sure that TCP port 7000 is open for sheepdog communication.

Note: At this time, the sheepdog init.d script is hard-coded to use /var/lib/sheepdog as the store directory. If you wish to use another directory, you will need to always start sheepdog using the 'sheep ' call.

$ sudo /etc/init.d/sheepdog start


$ sudo sheep /var/lib/sheepdog

The /var/lib/sheepdog will be where sheepdog will store objects. The directory must be on a filesystem with an xattr support. In case of ext3 or ext4, you need to add 'user_xattr' to the mount options.

$ sudo mount -o remount,user_xattr /var/lib/sheepdog
  1. Format sheepdog cluster
    $ collie cluster format --copies=3
    It is enough to run this command on just one machine. "--copies" specifies the number of default data redundancy. In this case, the replicated data will be stored on three machines.

Sheepdog has the back-end store to store the data object persistently on the node basis. Currently you have two choices, one is called Simple Store (default), the other Farm. Farm is supposed to provide advanced features such as cluster-wide snapshot, faster recovery, data de-duplication(not implemented yet) and so on.

You can choose to use Farm as the back-end store by following command:

$ collie cluster format -b farm --copies=3
  1. Check cluster state
    Following list shows that Sheepdog is running on 32 nodes.
    $ collie node list
     Idx    Node id (FNV-1a) - Host:Port
     0   0308164db75cff7e -
     1   03104d8b4315c8e4 -
     2   0ab18c565bc14aea -
     3   0c0d27f0ac395f5d -
     4   127ee4802991f308 -
     5   135ff2beab2a9809 -
     6   17bd6240eab65870 -
     7   1cf35757cbf47d7b -
     8   1df9580b8960a992 -
     9   29307d3fa5a04f78 -
    10    29dcb3474e31d4f3 -
    11    29e089c98dd2a144 -
    12    2a118b7e2738f479 -
    13    3d6aea26ba79d75f -
    14    42f9444ead801767 -
    15    562c6f38283d09fe -
    16    5dd5e540cca1556a -
    17    6c12a5d10f10e291 -
    18    6dae1d955ca72d96 -
    19    711db0f5fa40b412 -
    20    7c6b95212ee7c085 -
    21    7d010c31bf11df73 -
    22    82c43e908b1f3f01 -
    23    931d2de0aaf61cf5 -
    24    961d9d391e6021e7 -
    25    9a3ef6fa1081026c -
    26    b0b3d300fed8bc26 -
    27    b0f08fb98c8f5edc -
    28    b9cc316dc5aba880 -
    29    d9eda1ec29c2eeeb -
    30    e53cebb2617c86fd -
    31    ea46913c4999ccdf -

Create a VM image

  1. Create a 256 GB virtual machine image of Alice.

    $ qemu-img create sheepdog:Alice 256G # default as raw image
    $ qemu-img create -f qcow2 sheepdog:Alice 256G # you can enjoy all the qcow2 features
  2. You can also convert from existing KVM images to Sheepdog ones.

    $ qemu-img convert -t directsync ~/amd64.raw sheepdog:Bob
  3. See Sheepdog images by the following command.

    $ collie vdi list
    name        id    size    used  shared    creation time  object id
    Bob          0  2.0 GB  1.6 GB  0.0 MB 2010-03-23 16:16      80000
    Alice        0  256 GB  0.0 MB  0.0 MB 2010-03-23 16:16      40000

Boot the VM

  1. Boot the virtual machine with sheep daemon started on local node

    $ qemu-system-x86_64 sheepdog:Alice
  2. Boot the virtual machine without sheep daemon started on local node

    $ qemu-system-x86_64 sheepdog:

    This suppose you have one sheep started up on the node with the IP and port 7000

  3. Sheepdog supports local cache called object cache. Object cache caches data and vdi objects on the local node. It is at higher level than backend store. This extra cache layer translate gateway requests into local requests, largely reducing the network traffic and highly improve the IO performance. Dirty objects will be flushed to the cluster storage by 'sync' request from guest OS. You have to run latest QEMU to work with object cache. To enable the object cache, you can start QEMU by following command:

    $ qemu-system-x86_64 -drive file=sheepdog:Alice,cache=writeback


  1. Create a snapshot

    $ qemu-img snapshot -c name sheepdog:Alice
  2. After getting snapshot, a new virtual machine images are added as a not-current image.

    $ collie vdi list
      name        id    size    used  shared    creation time  object id
      Bob          0  2.0 GB  1.6 GB  0.0 MB 2010-03-23 16:16      80000
      Alice        0  256 GB  0.0 MB  0.0 MB 2010-03-23 16:21      c0000
    s Alice        1  256 GB  0.0 MB  0.0 MB 2010-03-23 16:16      40000
  3. You can boot from the snapshot image by specifying snapshot id

    $ qemu-system-x86_64 sheepdog:Alice:1

Cloning from the snapshot

  1. Create a Charlie's image as a clone of Alice's image.

    $ qemu-img create -b sheepdog:Alice:1 sheepdog:Charlie
  2. Charlie's image is added to the virtual machine list.

    $ collie vdi list
      name        id    size    used  shared    creation time  object id
      Bob          0  2.0 GB  1.6 GB  0.0 MB 2010-03-23 16:16      80000
      Alice        0  256 GB  0.0 MB  0.0 MB 2010-03-23 16:21      c0000
    s Alice        1  256 GB  0.0 MB  0.0 MB 2010-03-23 16:16      40000
      Charlie      0  256 GB  0.0 MB  0.0 MB 2010-03-23 16:23     100000

Shutdown Sheepdog

  1. Run the shutdown command on the one of the cluster machines.
    $ collie cluster shutdown

This command stops all the sheep processes on the cluster.

Test Environment

  • Debian squeeze amd64
Something went wrong with that request. Please try again.