RealmResults change listeners break sync queries

[knighty]

I have been working on re-architecting my app to make use of async queries. Unfortunately it seems like using any kind of change listener on realm results breaks sync queries in a subtle way.

For my sync queries, I'm pulling from an api, updating realm in another thread, passing the ids back and then loading the realm objects on the main thread from those ids. If there are no realm result change listeners registered elsewhere, this works fine. As soon as one is introduced, however, my queries on the main thread of those ids find no objects. There seems to be some kind of timing issue going on because if I put a breakpoint at the end of the transaction and leave it for a second, it all works fine.

Any idea what would be causing this timing issue with stale data?

Below is the code for my sync queries using Rx (some cruft chopped out for brevity)

Observable<List<Bookshelf>> network = dataAccess.api.getBookshelves(user_id)
                    .doOnNext(api_bookshelves -> {
                        Realm realm = Realm.getInstance(dataAccess.realmConfiguration);
                        realm.beginTransaction();
                        for (net.fimfiction.api.models.Bookshelf api_bookshelf : api_bookshelves) {
                            Bookshelf bookshelf = realm.where(Bookshelf.class).equalTo("id", api_bookshelf.id).findFirst();
                            if (bookshelf == null) {
                                bookshelf = realm.createObject(Bookshelf.class);
                            }
                            Bookshelf.fromApi(realm, bookshelf, api_bookshelf);
                        }
                        realm.commitTransaction();
                        realm.close();
//At this point, all the items have been supposedly committed to realm
                    })
                    .map(api_bookshelves -> {
                        int[] ids = new int[api_bookshelves.size()];
                        int i = 0;
                        for (net.fimfiction.api.models.Bookshelf bookshelf : api_bookshelves) {
                            ids[i++] = bookshelf.id;
                        }
                        return ids;
                    })
                    .materialize()
                    .subscribeOn(Schedulers.newThread())
                    .observeOn(AndroidSchedulers.mainThread())
                    .<int[]>dematerialize()
                    .map(new RealmMap<Bookshelf>(dataAccess.realm, Bookshelf.class));
//However, looking them up here (so within a very short amount of time and across a thread boundary) returns no results

RealmMap just handles mapping the int[] to a proper list of items.

Code for the async queries is as basic as any example. Just a simple query with a listener attached. I've verified it is definitely the listener and not the async query itself because if I just remove the listener it works fine again.

[knighty]

I've just done some more investigation because as I was pasting that code in I realised I didn't need to do this across threads (isn't that always the way with code) as I'd change my api to be async itself, and it's only a few items so quick enough on the ui thread. Anyway, by doing that it started working again, so it seems the issue is to do with short timing across 2 threads.

[knighty]

There's another bug (it seems) too which might warrant another issue. RealmResults do not get their change event fired on normal transactions, only with async transactions. Is this intended behaviour? Nowhere in the docs is this stated to be intended.

realm.executeTransaction(realm -> {
    //Transaction code
    }, null)

If null is removed to make a sync transaction, any realmresults are no longer updated after the transaction

edit: After running over the issues again I see this has already been brought up and is being addressed

[cmelchior]

Yes. It was intentional it didn't work for normal queries (mostly because of performance), but we forgot to document that restriction :(. In the mean time we have found a way to actually make it work effectively for both synchronous and asynchronous queries, so hopefully it will be a lot more intuitive really soon.

[cmelchior]

In the meantime I'll fold this issue into #1676 as it is effectively the same thing.

[knighty]

Awesome :) Any idea on the first issue? I appreciate it's pretty wishy washy because I'm using Rx which could maybe be related (don't think so though) and it's tricky timing issues across threads, but is there anything that happens to the realm when a RealmList change listener is added that could explain it at all? I can try to cook up some proper example code

[cmelchior]

I am not entirely sure about your code flow from your example as you only write to Realm on Scheduler.newThread(), but our listeners use the Looper to communicate, so if you have a lot of other events in the Looper queue it could potentially add a small delay. Maybe that is what you are seeing?

[knighty]

I'm not using a listener for this though. The objects are added to the realm, ids passed back and the objects are loaded again on the main thread with standard sync queries. It's when adding a listener anywhere else at all, totally unrelated, it suddenly breaks.

[cmelchior]

That sounds really strange. If you can reproduce it in an example project it would be very helpful.

[knighty]

I'll try and get around to it today because it certainly is strange! Thanks for the help.