Skip to content

Thread Safety

Robbie Hanson edited this page Dec 23, 2015 · 24 revisions

YapDatabase was designed with concurrency in mind. But that doesn't mean its impossible to shoot yourself in the foot. Arm yourself with knowledge so you never have any "accidents".


Connections, queues & deadlock

One of the powerful features of the YapDatabase architecture is that connections are thread-safe. That means that you can share a single YapDatabaseConnection amongst multiple threads. For example:

- (void)asyncSolveProblem:(Problem *)p forKey:(NSString *)key
{
    // Solve complex math problem on a background thread (in the thread pool)
    dispatch_queue_t concurrentQ = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
    dispatch_async(concurrentQ, ^{
        
        Answer *a = [complexMath solveProblem:p];
        
        // Now save the answer to the database.
        // The databaseConnection is thread-safe, so this is safe,
        // even though other threads may be using it simultaneously.
        [databaseConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
            
            [transaction setObject:a forKey:key inCollection:@"answers"];
        }];
    });
}

All connections have an internal serial dispatch queue. And all operations on a connection (such as executing a transaction) go through this internal serial queue. So its rather easy to conceptualize the nature of the thread-safety within a single connection: All transactions executed on connectionA will execute in a serial fashion.

The main thing to watch out for is executing a transaction within a transaction. This is not allowed, and will result in DEADLOCK :

- (void)deadlock1
{
    [databaseConnection readWithBlock:^(YapDatabaseReadTransaction *transaction) {
        
        id deadlock = [self deadlock2];
    }];
}

- (id)deadlock2
{
    __block id uh_oh = nil;
    [databaseConnection readWithBlock:^(YapDatabaseReadTransaction *transaction) {
        
        uh_oh = [transaction objectForKey:@"deadlock" inCollection:@"uh oh"];
    }];

    return uh_oh;
}

This is the most common reason for deadlock reports. Now that you understand the problem, I'm sure you can come up with multiple solutions. We'll present one such solution here. Not because its the best solution (you might like yours better), but rather because its one that tends to be thought of the least.

- (float)estimatedCostsForAddress:(Address *)addr
{
    __block float taxes = 0.0;

    [databaseConnection readWithBlock:^(YapDatabaseReadTransaction *transaction) {
        
        taxes += [self propertyTaxesForAddress:addr withTransaction:transaction];
        taxes += [self hoaFeesForAddress:addr withTransaction:transaction];
    }];

    return taxes;
}

- (float)propertyTaxesForAddress:(Address *)addr withTransaction:(YapDatabaseReadTransaction *)transaction
{
    NSString *key = [NSString stringWithFormat:@"%d", addr.zip];

    return [[transaction objectForKey:key inCollection:@"propertyTaxes"] floatValue];
}

Yup, it's safe to pass a transaction object to helper methods.

Remember:

  • This is just one example of a solution.
  • A transaction object should never be saved as an ivar. That won't work.

Object mutability vs thread-safety

As an Objective-C developer, you're likely familiar with the concept of mutable vs immutable objects. There are multiple classes in Apple's Foundation Framework that distinguish between the two:

  • NSArray vs NSMutableArray
  • NSDictionary vs NSMutableDictionary
  • NSSet vs NSMutableSet

And from Apple's Threading Programming Guide:

Immutable objects are generally thread-safe. Once you create them, you can safely pass these objects to and from threads. On the other hand, mutable objects are generally not thread-safe. To use mutable objects in a threaded application, the application must synchronize appropriately.

It's important to keep in mind that, although YapDatabase is thread-safe, the objects you're fetching from the database may not be. For example, consider the following code:

- (void)someMethodOnMainThread
{
    __block Person *person = nil;
    [connection readWithBlock:^(YapDatabaseReadTransaction *transaction){
        person = [transaction objectForKey:personId inCollection:@"persons"];
    }];
    
    // Accessing children array on main thread...
    for (Person *child in person.children)
    {
        [self addViewForChild:child];
    }
}

- (void)someMethodOnBackgroundThread
{
    [connection readWithBlock:^(YapDatabaseReadTransaction *transaction){

        Person *person = [transaction objectForKey:personId inCollection:@"persons"];
        
        // Modifying children array on background thread...
        [person.children addObject:newChild];
    }];
}

Looking at such a simple example, the mistake seems pretty obvious. But since YapDatabaseConnection is thread-safe, it can be easy to forget that thread-safety in one place doesn't extend to everything else.

But are we referring to the exact same person object? Or is each a different copy?

Recall that every connection has a cache. The cache is important for performance, and drastically reduces both trips to the disk, and the overhead of deserializing an object. Thus its highly likely that both the main thread and background thread are fetching the exact same Person instance. (Notice that both threads are using the same YapDatabaseConnection instance.) And so the background thread may be modifying the object while the main thread is simultaneously attempting to use it. This is no different than modifying an NSMutableArray from multiple threads without any locks around it.

If the objects you put in the database are mutable you must follow all the same rules & guidelines that you would for any other mutable object.

The recommended practice is to make copies of your objects before you modify them. For example:

- (void)someMethodOnMainThread
{
    __block Person *person = nil;
    [connection readWithBlock:^(YapDatabaseReadTransaction *transaction){
        person = [transaction objectForKey:personId inCollection:@"persons"];
    }];
    
    for (Person *child in person.children)
    {
        [self addViewForChild:child];
    }
}

- (void)someMethodOnBackgroundThread
{
    [connection readWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction){

        Person *person = [transaction objectForKey:personId inCollection:@"persons"];
        person = [person copy];
        [person.children addObject:newChild];
        
        [transaction setObject:person forKey:personId];
    }];
}

If you follow this simple guideline, you generally won't have to worry about thread-safety. Even when your objects are mutable.

If fact, if you follow this simple guideline, YapDatabase has additional performance optimizations you can enable to speed up your database. See the Object Policy article for additional details.


Thinking, asynchronous & synchronous

Entering a world of asynchronous programming means learning to think in terms of concurrent asynchronous operations. It's not easy at first. I cannot tell you how many times I shot myself in the foot. And just when I thought I had it all figured out, Apple went and created Grand Central Dispatch. All of a sudden I had dispatch_async() at my disposal, which meant I was doing all kinds of new async stuff. And shooting my foot in new and clever ways.

Just as tools like GCD make it easier to perform general async operations, YapDatabase makes it easy to access & update your database using async operations. And one of the most common mistakes I see people make is this one:

- (void)downloadImageForPost:(Post *)originalPost
{
    [[ImageDownloader sharedInstance] asyncDownloadImage:originalPost.imageURL
                                         completionBlock:^(NSString *dowloadedImageFilePath){

        [bgDatabaseConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction){

            // Update post
            Post *updatedPost = [originalPost copy]; <--- Uh-oh !!!
            updatedPost.imagePath = downloadedImageFilePath;
            
            [transaction setObject:updatedPost forKey:updatedPost.uuid inCollection:@"posts"];
        }];
    }];
}

What's wrong with this code? It's following the recommended practice stated above (make copies of your objects before you modify them)...

The code is very nearly perfect. We just forgot to put our "async thinking hat" on.

When you fetch an item from the database, you should think of that item as a snapshot in time. The object represents the state of that object at a particular commit. And you should keep in mind that there may be other commits happening in the background that might just store an updated version of that object to the database. For example, imagine the class has another method like this:

- (void)markPostAsRead:(NSString *)postId
{
    [bdDatabaseConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction){

        Post *post = [transaction objectForKey:postId inCollection:@"posts"];
        post = [post copy];
        post.isRead = YES;

        [transaction setObject:post forKey:postId inCollection:@"posts"];
    }];
}

So what will happen when we do this?:

- (UITableViewCell *)tableView:(UITableView *)sender cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
    __block Post *post = nil;
    [databaseConnection readWithBlock:^(YapDatabaseReadTransaction *transaction) {    
        post = [[transaction ext:@"order"] objectAtIndex:indexPath.row inGroup:@"postsFromFriends"];
    }];

    // create and configure cell using post...

    // since we're display the post, we can mark it as read now
    if (!post.isRead) {
        [self markPostAsRead:post.uuid];
    }

    // download the image if needed
    if (post.imageURL && !post.imagePath) {
        [self downloadImageForPost:post];
    }

    return cell;
}
  • The first async commit will set post.isRead to YES
  • And the second async commit will accidentally undo the first, by using an old version of the post object

Luckily the fix is easy. Before editing an object in the database, be sure to grab the latest revision of the object within the read-write transaction.

(Recall that there can only be a single read-write transaction at any one time. So if you follow this guideline, you'll always be sure to update the most recent revision of the object.)

- (void)downloadImageForPost:(Post *)originalPost
{
    [[ImageDownloader sharedInstance] asyncDownloadImage:originalPost.imageURL
                                         completionBlock:^(NSString *dowloadedImageFilePath){

        [bgDatabaseConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction){

            // Update post (be sure to grab latest revision of object)
            Post *post = [transaction objectForKey:originalPost.uuid inCollection:@"posts"];
            post = [post copy];
            post.imagePath = downloadedImageFilePath;
            
            [transaction setObject:post forKey:post.uuid inCollection:@"posts"];
        }];
    }];
}
You can’t perform that action at this time.