Support for asynchronous writes

[astigsen]

What are the current thoughts on adding a shortcut syntax for doing async transactions (where operations happen on a background thread)?

Here is the current recommended way of doing it manually (from the docs):

// Query from any thread
dispatch_async(dispatch_queue_create("background", nil)) {
  let realm = try! Realm()
  let theDog = realm.objects(Dog).filter("age == 1").first
  try! realm.write {
    theDog!.age = 3
  }
}

It is very flexible, but it does open up the opportunity to accidentally forgetting to get a new Realm, and when working with non-default realms, you have to be careful to pass it the right configuration (also, you probably want it wrapped in an autoreleasepool, like in some of the later examples).

Would it make sense to offer a simpler version that ensured that you get the right realm (and that it get released afterwards)? Maybe something like this:

// Query and update on background thread
realm.async { realm in
  let theDog = realm.objects(Dog).filter("age == 1").first
  try! realm.write {
    theDog!.age = 3
  }
}

Or even simpler if you know that you need to write:

// Write via background thread
realm.async_write { realm in
  let theDog = realm.objects(Dog).filter("age == 1").first
  theDog!.age = 3
}

This kind of shortcut syntax would obviously need a bit of thought in regard to how you would then handle errors (can't open realm, commit fails), but offering the option to supply a closure called on error might satisfy that.

[jpsim]

To me, the value add is much clearer for async write transactions than with read transactions. One big question is what queue quality to pick since the point of such an API would be to decide this for users so they don't have to. The beauty of the current thread-agnostic API is its flexibility, which this would trade against a bit less verbosity.

As discussed in private, I'd see a family of async write APIs, which could use a concurrent, low priority queue:

extension Realm {
  func asyncWrite(writeBlock: Realm -> Void) throws
  func asyncWrite<T: Object>(object: T, writeBlock: (Realm, T) -> Void) throws
  func asyncWrite(objects: Object..., writeBlock: (Realm, [Object]) -> Void) throws
  func asyncWrite<T: Object>(objects: RealmCollection<T>, writeBlock: (Realm, RealmCollection<T>) -> Void) throws
}

We could also have a queue parameter with a carefully default value so that if users don't care, they use our default, and if they do care, they can provide their own.

I'll mark this as a backlog priority for now, and we can continue discussing.

[astigsen]

func asyncWrite(writeBlock: Realm -> Void) throws

Does it make sense for a method like that to throw? If an error happens, it is likely to happen at a later time and on a different thread. Would it not make more sense for it to take a closure argument for error handling?

[jpsim]

The method wouldn't throw, the closure would. My sample is incorrect.

[JadenGeller]

@jpsim The asyncWrite function must provide a completion block so that asynchronous errors (on commit after the write block is executed) can be handled. Since most users would probably ignore the error in the completion block, we ought to provide a default implementation that aborts. Unfortunately we are unable to show variant may fail (like the try! annotates with synchronous write), but this is a limitation of Swift.

I propose the following functions for the no-parameter and single-object-parameter cases. I agree that the multi-object-parameter variants are also useful, but I would first like to discuss the simpler cases.

public func asyncWrite(onQueue queue: dispatch_queue_t = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), do block: Realm -> Void) {
    asyncWrite(onQueue: queue, do: block, then: { error in
        if let error = error { try! { throw error }() }
    })
}
public func asyncWrite(onQueue queue: dispatch_queue_t = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), do block: Realm -> Void, then completion: ErrorType? -> Void) {
    fatalError("Unimplemented")
}
public func asyncWrite<T: Object>(onQueue queue: dispatch_queue_t = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), with object: T, do block: (Realm, T) -> Void) {
    asyncWrite(onQueue: queue, with: object, do: block, then: { error in
        if let error = error { try! { throw error }() }
    })
}
public func asyncWrite<T: Object>(onQueue queue: dispatch_queue_t = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), with object: T, do block: (Realm, T) -> Void, then completion: ErrorType? -> Void) {
    fatalError("Unimplemented")
}

Note that the completion block is provided in an overload rather than a default parameter so that users can use trailing-closure syntax in the non-handling case.

It's not straightforward how this API ought to translate to Objective-C. If we wish to provide "default" arguments for queue in the Objective-C API, we have to provide 2x as many functions, and there are already quite a lot. If we do, I think something like this might be reasonable, but I'm not very happy with the number of "overloads".

- (void)asyncTransaction:(void(^)(RLMRealm *))block;
- (void)asyncTransactionOnQueue:(dispatch_queue_t)queue executeBlock:(void(^)(RLMRealm *))block;
- (void)asyncTransactionWithObject:(RLMObject *)object executeBlock:(void(^)(RLMRealm *, RLMObject *))block;
- (void)asyncTransactionOnQueue:(dispatch_queue_t)queue withObject:(RLMObject *)object executeBlock:(void(^)(RLMRealm *, RLMObject *))block;

- (void)asyncTransaction:(void(^)(RLMRealm *))block completion:(void(^)(NSError * _Nullable));
- (void)asyncTransactionOnQueue:(dispatch_queue_t)queue executeBlock:(void(^)(RLMRealm *))block completion:(void(^)(NSError * _Nullable));
- (void)asyncTransactionWithObject:(RLMObject *)object executeBlock:(void(^)(RLMRealm *, RLMObject *))block completion:(void(^)(NSError * _Nullable));
- (void)asyncTransactionOnQueue:(dispatch_queue_t)queue withObject:(RLMObject *)object executeBlock:(void(^)(RLMRealm *, RLMObject *))block completion:(void(^)(NSError * _Nullable));

And now you can probably see why I left off discussing the other variants for now…! I'd appreciate feedback on whether this direction is reasonable.
//cc @austinzheng @tgoyne @TimOliver @mrackwitz

[bigfish24]

@JadenGeller I think it was suggested somewhere that we would have a default background queue similar to the default file path, where it is defined globally and not tied to the methods themselves. Is this too limiting?

[tgoyne]

What is the use-case for letting the user specify the queue to use? If they need it to be on a specific queue for the sake of synchronizing access with other things they can just use a dispatch_sync() within the write block. Being able to set the priority seems not all that important.

For the sake of not having an explosion of overloads I'd be inclined to have just asyncWriteWithObjects:block: and aysncWriteWithObjects:block:completion: and have the user just pass nil for the first parameter if they don't want to hand anything over.

We could even skip the completion block entirely and tell people to just call commit() directly within the block if they want to handle errors from it.

[bigfish24]

If we skip the completion block then we would likely need to adjust API to something like handoverObjects: since the block wouldn't be wrapped in a transaction automatically right?

[tgoyne]

No, it would still be automatically wrapped in a transaction. Calling commit() from within a transaction block is allowed.

[bigfish24]

Ok, well my preference is on having a completion block since it is more obvious how to handle the error.

[austinzheng]

Could you use a variadic parameter (e.g. foo(x: Object...))? Sorry, I'm still reviewing the thread.

[JadenGeller]

We definitely could, but this would require the user to downcast as in the 1st example: #3136 (comment)

[austinzheng]

Yeah, I think you're right. There's no way to both accept an arbitrary number of arguments and keep type information at the same time.

[JadenGeller]

@austinzheng @tgoyne Is it fairly easy to integrate GYB into our build system? Is this something we'd even necessarily want to do?

[austinzheng]

I think Thomas was being facetious, but if he wasn't GYB is just a Python script we can steal from the Swift repo and invoke at build time.

[tgoyne]

If we do go with the code-gen route we'd probably want to just commit the end result and have regenerating it be a manual step to avoid having to make it work when building from the podspec. I also would just write like a five line shell/ruby/whatever script to generate the methods in a separate file as an extension on Realm. Could even use boost.PP and the C preprocessor! _{(don't do this)}

Regardless of how we did it it'd be a giant nightmare, but a giant nightmare for us that makes the API a lot more pleasant for the users may be worth it.

[jpsim]

Speaking with @JadenGeller about this just now, I'd like to add to the bikeshedding:

1. Why not avoid adding new ways to handle errors and reuse the ones we already have?
   By making async read transactions (though they have little value on their own), we
   can let users reuse the write methods and associated error handling that they're
   already familiar with.
2. By forgoing variadic arguments, we can use SequenceType and preserve the common
   denominator type. In other words, pass in [Model1(), Model1()] and the block vends
   an objects of type [Model1]; pass in [Model1(), Model2()] and the block vends an
   objects of type [Object]. Same thing for RealmCollectionType<T>.

Here's a small Swift Playgrounds that can serve as a proof of concept:

import Foundation

class Object: NSObject {}
class Realm {}

class Model1: Object {}
class Model2: Object {}

extension Realm {
    func async(block: (Realm) -> Void) {
        block(self)
    }

    func async<S: SequenceType where S.Generator.Element: Object>(objects: S, block: (Realm, S) -> Void) {
        block(self, objects)
    }
}

let realm = Realm()
let array = [Model1(), Model1()]
let set: Set = [Model1(), Model1()]
let polymorphic = [Model1(), Model2()]

realm.async(array) { realm, objects in
    // objects: Array<Model1>
}
realm.async(set) { realm, objects in
    // objects: Set<Model1>
}
realm.async(polymorphic) { realm, objects in
    // objects: Array<Object>
}

[JadenGeller]

@jpsim We won't be able to construct an S from the new objects on the other thread.

That is to say that the API would need be something like:

func async<S: SequenceType where S.Generator.Element: Object>(objects: S, block: (Realm, AnySequence<S.Generator.Element>) -> Void)

[JadenGeller]

I definitely prefer the async read API to the asyncWrite API, so +1 on that!

[JadenGeller]

Elaborating on @jpsim's suggestion, this API design doesn't require us to provide a completion handler. Since the only throwing call during handover is creating a Realm instance on the new thread, and since this will never fail (right?), we don't need to provide an error path.

[JadenGeller]

What do y'all imagine the behavior should be if an Object subclass that isn't yet managed by Realm is handed over? Probably throw an exception, but we could reasonably create a copy of the object instead.

[bigfish24]

@JadenGeller we should have a completion handler so users can build reactive code from it

[JadenGeller]

@bigfish24 I don't think a completion handler is necessary to build reactive code in this case. It completes when their code completes.

realm.async(handingOver: object) { realm, object in
    // do stuff
    arbitraryUserCallback()
}

A completion handler is only necessary if we perform some operation after their block completes.

[bigfish24]

If the block is wrapped in a write transaction, you might want to do something after that.

[JadenGeller]

@jpsim and I were discussing not wrapping the block in a write transaction since it'd mean we'd retain the same try error handling for the write block.

[bigfish24]

Ah I didn't catch that, so its just handover, with the user needing to do a write transaction in the block manually

[JadenGeller]

It's reasonable that a user might want to do an async read, and this API allows that. Even if that's uncommon, it seems like a good idea to allow users to use the same familiar write function they already know how to use rather than introduce a semantically different alternative.

realm.async(handingOver: object) { realm, object in
    try! realm.write {
        object.foo = "bar"
    }
    // if you want to do anything after completion, do it here
}

[JadenGeller]

@jpsim Actually, we can provide a polymorphic implementation if we constrain to RangeReplaceableCollectionType instead of SequenceType since the former requires both init() and append(_:). 👍

[santhoshs5]

Hey guys ,

I am late to this party and newbie to Realm

I have created a signleton class having following method to write but it crashes at times because incorrect thread access

Let me know what I am doing wrong here.

 func save<T:Object>(_ realmObject:T) {
        let backgroundQueue = DispatchQueue(label: ".realm", qos: .background)
        backgroundQueue.async {
            let realm = try! Realm()
           try! realm.write {
                realm.add(realmObject)

            }
        }
        
   }

[jpsim]

Hi @santhoshs5, even though we've released support for thread-safe references, Realm objects are still otherwise thread-confined. To modify your example:

func save<T: Object>(_ realmObject: T) {
  let backgroundQueue = DispatchQueue(label: ".realm", qos: .background)
  let ref = ThreadSafeReference(to: realmObject)
  backgroundQueue.async {
    let realm = try! Realm()
    guard let obj = realm.resolve(ref) else { return } // object was deleted
    try! realm.write {
      realm.add(obj)
    }
  }
}

Please refer to our release post on thread-safe references to learn more: https://realm.io/news/obj-c-swift-2-2-thread-safe-reference-sort-properties-relationships/

---

I'll be closing this ticket now as we have a mechanism for asynchronously passing thread-safe references in Realm and performing write transactions, and we have a separate ticket open to track making convenience APIs around it in #4477.