Skip to content

Guide: Replication

Jump to bottom
Jens Alfke edited this page Aug 5, 2012 · 18 revisions

TABLE OF CONTENTS

8. Replication

Here we are: this is the feature that probably induced you to want to use TouchDB. You have progressed well through the info-dump, Grasshopper, and are ready to learn the secret techniques of document syncing.

Replication is conceptually simple — "Take everything that's changed in database A and copy it over to database B", but it comes with a sometimes-confusing variety of options:

  • Push vs Pull. This is really just a matter of whether A or B is the remote database.
  • Continuous vs. one-shot. A one-shot replication proceeds until all the current changes have been copied, then finishes. A continuous replication keeps the connection open, idling in the background and watching for more changes; as soon as any happen, it copies them. (TouchDB's replicator is aware of connectivity changes, so if the device goes offline the replicator will watch for the server to become reachable again, and then reconnect.)
  • Persistent vs. non-persistent. Non-persistent replications, even continuous ones, are forgotten after the app quits. Persistent replications are remembered in a special _replicator database. This is most useful for continuous replications: by making them persistent, you ensure they will always be ready and watching for changes, every time your app launches.
  • Filters. Sometimes you only want particular documents to be replicated, or you want particular documents to be ignored. To do this you can define a filter function. The function simply takes a document's contents and returns true if it should be replicated.

Creating A Replication

Replications are represented in CouchCocoa by objects, of class CouchReplication (for non-persistent ones) or CouchPersistentReplication. We'll focus on the persistent kind first, as they're more commonly used. You get them by asking the local CouchDatabase, calling replicationFromDatabaseAtURL: or replicationToDatabaseAtURL:. Or you can get a bulk discount by calling replicateWithURL: to set up a bidirectional replication:

NSArray* repls = [self.database replicateWithURL: newRemoteURL exclusively: YES];
self.pull = [repls objectAtIndex: 0];
self.push = [repls objectAtIndex: 1];

The exclusively: YES option will seek out and remove any pre-existing replications with other remote URLs. This is useful if you only sync with one server at a time and just want to change the address of that server.

It's not strictly necessary to keep references to the replication objects, but you'll need them if you want to monitor their progress.

Monitoring Replication Progress

A replication object has several properties you can observe to track its progress. The most useful are:

  • .completed — the number of documents copied so far in the current batch
  • .total — the total number of documents to be copied
  • .error — will be set to an NSError if the replication fails
  • .mode — an enumeration that tells you whether the replication is stopped, offline, idle or active. (Offline means the server is unreachable over the network. Idle means the replication is continuous but there is currently nothing left to be copied.)

Generally you can get away with just observing .completed:

[self.pull addObserver: self forKeyPath: @"completed" options: 0 context: NULL];
[self.push addObserver: self forKeyPath: @"completed" options: 0 context: NULL];

Your observation method might look like this:

- (void) observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object 
                         change:(NSDictionary *)change context:(void *)context
{
    if (object == pull || object == push) {
        unsigned completed = pull.completed + push.completed;
        unsigned total = pull.total + push.total;
        if (total > 0 && completed < total) {
            [self showProgressView];
            [progressView setProgress: (completed / (float)total)];
        } else {
            [self hideProgressView];
        }
    }
}

Here progressView is a UIProgressView that shows a bar-graph of the current progress. The progress view is only shown while replication is active, i.e. when total is nonzero.

Don't expect the progress indicator to be completely accurate! It may jump around because the .total property changes as the replicator figures out how many documents need to be copied. And it may not advance smoothly, because some documents may take much longer to transfer than others if they have large attachments. But in practice it seems accurate enough to give the user an idea of what's going on.

One-Shot Replications

In some case you don't want a persistent replication; maybe you just want to pull (or push) once. Or maybe you want to control exactly when replication happens instead of letting TouchDB schedule it. In that case you can create non-persistent replications:

CouchReplication* pull = [self.database pullFromDatabaseAtURL: remoteURL];

or

CouchReplication* push = [self.database pushToDatabaseAtURL: remoteURL];

Note that these create a different class of object, CouchReplication as opposed to CouchPersistentReplication. By historical accident these classes are unrelated (one isn't a superclass of the other), but they have almost the same API.

Filtered Replications

It's pretty common to want to replicate only a subset of documents, especially when pulling from a huge cloud database down to a limited mobile device. For this purpose, TouchDB (like CouchDB) supports user-defined filter functions in replications. A filter function is registered with a name; it takes a document's contents as a parameter and simply returns true or false to indicate whether it should be replicated.

Filtered Pull

Filter functions are run on the source database. In a pull, that would be the remote CouchDB server, so that server must have the appropriate filter function. CouchDB filters are written in JavaScript (usually) and stored in design documents. The API documentation on the CouchDB wiki has details. This does mean that if you don't have admin access to the server, you are restricted to the set of already existing filter functions.

To use an existing remote filter function in a pull replication, set the replication's filter property to the filter's full name, which which is the design document name, a slash, and then the filter name:

pull.filter = @"grocery/sharedItems";

Filtered Push

During a push, on the other hand, the filter function runs locally in TouchDB. As with map/reduce functions, the filter function is nominally associated with a design document, but has to be specified at runtime as a native block pointer. Here's an example of defining a filter function that passes only documents with a "shared" property with a value of true:

CouchDesignDocument* design = [database designDocumentWithName: @"grocery"];
[design defineFilterNamed: @"sharedItems"
                    block: ^(NSDictionary* doc) {
						return [[doc objectForKey: @"shared"] booleanValue];
				    }];

This function can then be plugged into a push replication using its full name:

push.filter = @"grocery/sharedItems";

Parameterized Filters

Filter functions can be made more general-purpose by taking parameters. For example, a filter could pass documents whose "owner" property has a particular value, allowing the user name to be specified by the replication. That way there doesn't have to be a separate filter for every user.

To specify parameters in CouchCocoa, set the filterParams property of the replication object. Its value is a dictionary that maps parameter names to values. The dictionary must be JSON-compatible, so the values can be any type allowed by JSON.

CouchDB-based filter functions access the parameters through the HTTP request object (the query property of the req argument to the function).

Unfortunately TouchDB doesn't yet support passing parameters to filter functions (issue #132) so you can't use this in push replications yet.

Further Reading

Clone this wiki locally