A free software (AGPL) alternative to Dropbox
Perl Perl6 Shell
Latest commit aa3dfa6 Jan 3, 2015 @hoelzro Update README

README.pod

NAME

SaharaSync - Sync freely

VERSION

version 0.01

DESCRIPTION

NOTE I have not been working on this software actively for quite some time; in early 2012, my need for it disappeared, and since then, a lot of great programs that fulfill the same need have sprung up. I look at the code every so often to make sure it basically works, but active development has ceased. If you have interest in working on it, however, please let me know!

Sahara Sync is a file-synchronization program, simliar to web services such as Dropbox. The main difference is that with Sahara Sync, you are in control of your data and the code that handles them.

OVERVIEW

Sahara Sync has two components: the host daemon and the client daemon. If you're connecting to an existing Sahara Sync installation, you only need to worry about the client daemon.

THE HOST DAEMON

The host daemon (also known as hostd) is a process that runs on a server and is responsible for storing your files and information about them. It also communicates changes as they happen to the connected client daemons.

For more information, see SaharaSync::Hostd.

CLIENTD

The client daemon (also known as clientd) is a process that runs on each computer that wants to synchronize its files. The client daemon detects changes as they happen on that computer and sends them to a host daemon; it also receives messages from the host daemon when another client daemon has changed a file.

INSTALLATION

To get started using Sahara Sync, you need to install it if you haven't already. I'll continue this tutorial as if you need to set up both a host daemon and a client daemon; feel free to ignore the host daemon parts if you already have one to connect to.

INSTALLING HOSTD

To install hostd on a server, first create an account for hostd to run as. This isn't strictly necessary, but it's good practice:

rob@localhost $ sudo useradd saharasync
rob@localhost $ sudo -u saharasync -i
saharasync@localhost $

Also, make sure you have Perl 5.10 or greater installed on your server. Most servers have Perl already, so I won't cover that.

Because Sahara Sync has yet to be distributed as a package, you'll need to install it via CPAN. And in the interest of not polluting your system Perl with a smorgasbord of dependencies, you should probably install them within a perlbrew environment (for an explanation of perlbrew, please visit http://perlbrew.pl). So install perlbrew:

saharasync@localhost $ curl -kL http://install.perlbrew.pl | bash -
saharasync@localhost $ echo "source ~/perl5/etc/bashrc" >> .bashrc # change for your
                                                                   # shell
saharasync@localhost $ source ~/perl5/etc/bashrc

Also, you should install cpanminus, an alternative CPAN client. You could use the default CPAN client, but cpanminus won't bother you with a bunch of questions:

saharasync@localhost $ perlbrew install-cpanm

Now you'll need to build a Perl environment in which to install Sahara Sync and its dependencies. As of this writing, Perl 5.14.2 is the most current, so we'll install that:

saharasync@localhost $ perlbrew install --as saharasync perl-5.14.2

This builds a fresh Perl interpreter and runs its test suite, so this will take about thirty minutes.

Now you can install Sahara Sync! Go grab a coffee or something; this will take a while!

saharasync@localhost $ cpanm SaharaSync

INSTALLING CLIENTD

Since the client daemon and the host daemon are both in the same distribution, you can follow the instructions above, starting with the part about installing perlbrew. Just make sure you install everything as the user that will actually be using Sahara Sync!

SETUP

Setting up the daemons is pretty easy!

SETTING UP HOSTD

Setting up the host requires you to create a configuration file. The configuration file has three sections: storage options, server behavior options, and logging options. The Sahara Sync distribution comes with an example configuration in host.yaml; let's go over each section individually.

Storage

The storage section looks something like this:

storage:
  type: 'DBIWithFS'
  dsn: 'dbi:Pg:dbname=sahara'
  storage_path: '/tmp/sahara'

The 'type' parameter specifies which storage backend to use. For now, only 'DBIWithFS' is supported. The remaining parameters are parameters to the storage backend. In the case of DBIWithFS, 'dsn' specifies the database connection string (which we'll cover in a bit), and 'storage_path' specifies where the actual files will be located on disk. The storage_path should be pretty straightforward; just pick a directory, create it, and change the value in the configuration file. The dsn is a little more complicated, as we need to set up a database and initialize it with the Sahara Sync schema. The DBIWithFS backend currently supports SQLite, MySQL, and PostgreSQL; let's cover them individually.

SQLite

SQLite is pretty easy to set up; just pick a spot for it and go! I'll use /tmp/sahara-sync.db as an example:

saharasync@localhost $ sqlite3 -init schema.sqlite /tmp/sahara-sync.db

Done! Now, for the dsn option, it should look something like this:

dbi:SQLite:dbname=/tmp/sahara-sync.db

See the DBD::SQLite documentation for more information on the dsn when using SQLite.

MySQL

MySQL is a little more complicated to set up, but for the purposes of this document, I'll assume you have a server running on localhost, and that you'll be storing your data in a database called 'sahara'. First, we need to install the MySQL adapter for DBI, the library that Perl uses to access databases:

saharasync@localhost $ cpanm DBD::mysql

Now, let's create the database:

saharasync@localhost $ echo 'create database sahara' | mysql -u $MY_USER --password=$MY_PASSWD --batch

And finally, initialize the schema:

saharasync@localhost $ mysql -u $MY_USER --password=$MY_PASSWD --batch sahara < schema.mysql

Done! Now, for the dsn option, it should look something like this:

dbi:mysql:database=sahara

The DBIWithFS backend also supports a 'user' parameter and a 'password' parameter; these should be set to a MySQL user and its password.

See the DBD::mysql documentation for more information on the dsn when using MySQL.

PostgreSQL

PostgreSQL is about as complicated to set up as MySQL; like MySQL, I will assume you have a server running on localhost, and that you'll be storing your data in a databaes called 'sahara'. Also like MySQL, we need to install the PostgreSQL adapter for DBI:

saharasync@localhost $ cpanm DBD::Pg

Now, let's create the database:

saharasync@localhost $ createdb -U $PGUSER --password sahara

And finally, initialize the schema:

saharasync@localhost $ psql -X -f schema.psql -U $PGUSER --password sahara

Done! Now, for the dsn option, it should look something like this:

dbi:Pg:database=sahara

The DBIWithFS backend also supports a 'user' parameter and a 'password' parameter; these should be set to a PostgreSQL user and its password.

See the DBD::Pg documentation for more information on the dsn when using PostgreSQL.

Creating your user

Unfortunately, Sahara Sync doesn't currently expose an interface for creating users, so you'll need to do it directly via SQL. Log in to your database of choice and submit the following statement:

INSERT INTO users (username, password) VALUES ('USERNAME', 'PASSWORD');

Server

The server section isn't specified in the example configuration file, but if it were, the defaults would look like this:

server:
  host: 0.0.0.0
  port: 5982
  disable_streaming: false

The 'host' option specifies the bind address. You won't usually touch this, unless you intend to put the host daemon behind a proxy, or you only intend to access the host via an SSH tunnel.

The 'port' option specifies which port the host daemon will listen on. The default is probably fine.

The 'disable_streaming' option specifies whether or not the host daemon should allow for the streaming of changes. When disabled, clients must poll the host daemon for changes, which is less efficient. However, streaming changes aren't supported in every environment.

Logging

The logging section of the host daemon configuration is a series of configuration objects. Each object has a 'type' key, which specifies what kind of logger to create. These types are provided to Log::Dispatch, so if you specify 'File' as your logger's type, a Log::Dispatch::File object will be used as one of your loggers. You may specify as many loggers as you like; the daemon will run as long as at least one of them is loaded successfully. If you specify the 'mandatory' key in a logger's configuration, the daemon will fail to run if that logger cannot be loaded. Here's an example log section that appends output to a file called /var/log/saharasync/hostd.log:

log:
  -
    type: File
    min_level: info
    newline: true
    mode: '>>'
    filename: '/var/log/saharasync/hostd.log'

SETTING UP CLIENTD

If you put your hostd behind an SSL reverse proxy, you'll need to install the Net::SSLeay module.

RUNNING THE DAEMONS

RUNNING HOSTD

RUNNING CLIENTD

DEVELOPMENT

GETTING HELP

IRC

We have a channel on irc.freenode.net: #saharasync.

MAILING LIST

We also have a mailing list! users@saharasync.net

WEBSITE

We even have a website! http://saharasync.net

FAQ

AUTHOR

Rob Hoelz <rob@hoelz.ro>

COPYRIGHT AND LICENSE

This software is Copyright (c) 2015 by Rob Hoelz.

This is free software, licensed under:

The GNU Affero General Public License, Version 3, November 2007

BUGS

Please report any bugs or feature requests on the bugtracker website https://github.com/hoelzro/sahara-sync/issues

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.