Backbone-associations much *slower* than backbone-relational in my use case!

[ronen]

Hello,

I’ve been using backbone-relational and my app has been a bit sluggish. Since backbone-relational has the reputation of being slow and since backbone-associations promises to be faster, I figured I’d switch over. (The other potential benefits such as lack of a global store were certainly appealing too.) But after making the switch…

Much to my dismay, with backbone-associations my app runs about 5 times slower! :(

I’ve amended your jsperf suite to dummy up a subgraph of the same rough complexity as my use case — http://jsperf.com/backbone-associations-speed-suit/10 — which demonstrates the performance hit using backbone-associations, about 4 times slower on my platform. (See the Info section of the suite page for a brief description of the additional graph structure.)

Here are some thoughts, for what they're worth:

• I haven’t looked into it in great detail, but did a small bit of monkey-patch instrumentation in my app. This leads me to think that a contributing factor might be simply the sheer number of events that backbone-associations triggers. I’m looking at the time to initially load, configure, & render an object graph that’s fetched from the server; as a gauge fo complexity, for one medium-sized use case, the graph ends up with 3152 models in it. Using backbone-relational, the load/configure/render takes roughly 10 seconds and triggers 60,742 model events, whereas using backbone-associations it takes close to 1 minute and triggers 197,440 events.
• Another consideration is that until now, the jsperf suite has been comparing only the performance for set(). In http://jsperf.com/backbone-associations-speed-suit/10, the first lines of the test are doing mostly set()s — if you comment out the last line, backbone-associations continues to run faster. But the last line, runExperiments(…) makes a large number of of calls to add() as well as additional calls to get() and set(), and that’s where backbone-associations slows way down.
• A third possible consideration is that in this example, the graph has a bunch of cross-references (i.e. isn’t simply a tree structure), and has perhaps an extra level of depth from the root.

I don’t have any great insights or suggestions to propose, I just figured I’d bring this to your attention. If you have any insights I’d be glad to hear them!

Also cc’ing @PaulUithol and @philfreo of the backbone-relational team, it seems only fair to bring this to their attention too. (And likewise I’d be happy to hear any insights from you guys!)

[dhruvaray]

@ronen : I have mades some fixes to code which should help. In general, I should have been more observant about performance while adding features in the last couple of releases :(

The #s with the edge version of BB-Associations for your use case are @ http://jsperf.com/bb-associations-add-op/1

Recently, we introduced eventing with wild character (*) and nested-change events. If you are not interested in those kinds of events, in the edge version of BB-Associations, you can switch them off. The performance characteristics without those events bubbling up are @ http://jsperf.com/bb-associations-add-op/2

Can you try the edge version with your code-base and let me know how it goes?

[ronen]

@dhruvaray thanks for the quick response. I'll try it out today.

As an aside, the ability to switch off wildcard and nested-changed events does seem handy -- but i'd imagine it'd be even more handy if they could be switched on per relation, so that they could be enabled where needed without paying the cost everywhere. (This just mouthing off the top of my head -- I of course have no experience with them at all.)

[ronen]

@dhruvaray i tried the edge code in my app, it's now no longer 5x slower -- but it's not any faster. with

 Backbone.Associations.EVENTS_BUBBLE = false
 Backbone.Associations.EVENTS_WILDCARD = false
 Backbone.Associations.EVENTS_NC = false

my informal timing tests case are showing bb-assocaitions to be between the same speed and 10% slower than bb-relational. If I turn those all on, it seems to be about 20% slower.

One thing that might be contributing to that is that using bb-relational i'm taking advantage of explicit reverse relations, whereas in bb-associations i'm not using explicit reverse relations. instead i added some extra initialization/event code to manually set up back references when a forward reference gets created; it's possible that's causing some non-trivial work. (I didn't mention earlier, but: in my app creating explicit reverse relations by explicitly declaring both sides, as discussed in #51, causes it to be so slow as to be entirely unusable. Tomorrow I'll try to update the jsperf to reproduce that, and if i succeed i'll open a separate issue for it.)

[dhruvaray]

Thats' interesting. There might be other elements at play in your app - perhaps?

The jsperf results for the test case you gave yesterday show the following improvements

http://jsperf.com/bb-associations-add-op/1 : ~35-40% faster than bb-rel //All events in full glory
http://jsperf.com/bb-associations-add-op/2 : ~45-50% faster than bb-rel //No NC events and wild-card events
http://jsperf.com/bb-associations-add-op/3 : ~50-55% faster than bb-rel //Bubble events off totally

It will be really nice if you can abstract and share the jsperf test-case for the reverse relations.

Just as a note : You may be already doing this, but can you confirm that you are not loading both BB-Rel and BB-Associations simultaneously in your app.

[ronen]

Indeed there might be other elements at play. I'm trying to make the minimal changes possible between using BB-Associations and BB-Relational to make it a fair comparison, but it's hard to do that 100% because of the lack of reverse relations.

Anyway after some more tweaking (including the reverse-relations workaround) I'm seeing perhaps 10% speedup for the set portion of my app, but the heavy add portion of my app is still running slower. Will see if I can isolate something for jsperf.

Regarding reverse relations: I use them "densely": i.e. every Many relation has a corresponding reverse One relation. Even after fixing the infinite recursion (#91) it's still much slower. Part of that might simply be because I've now doubled the number of relations? Or maybe there's something else going on...? My first attempt at abstracting that in a jsperf wasn't showing the slowdown. I'll see if I can come up with something.

And yes, confirming I'm not loading both BB-Rel and BB-Associations simultaneously -- for my comparisons I switch back and forth between two different branches of my repo, one that includes only BB-Rel.js and the other only BB-Assoications.js

[dhruvaray]

I hope you are have set Backbone.Associations.EVENTS_BUBBLE = false. As far as I can remember, BB-rel does not bubble events beyond the immediate level.

I wrote a quick jsfiddle with add/set and rev-rel

Here's a jsfiddle with add + rev-rel (standard bb-rel example) : http://jsperf.com/rev-rel/2
Here's a jsfiddle with set + rev-rel (standard bb-rel example) : http://jsperf.com/rev-rel/3

I am not as experienced with bb-rel reverse relations. So please do modify if you want to write out a more representative example.

I am yet not clear why we need the check you put in PR (#91). So I am going to keep it open for a while

[dhruvaray]

As an aside, the ability to switch off wildcard and nested-changed events does seem handy -- but i'd imagine it'd be >even more handy if they could be switched on per relation, so that they could be enabled where needed without >paying the cost everywhere. (This just mouthing off the top of my head -- I of course have no experience with them >at all.)

You can achieve this by passing {silent:true} in the options param for set/add operations - at an instance level granularity.

[ronen]

FYI with #92 giving an implementation of BB-Associations with "native support" for reverse keys, I will see if I can revisit the timing comparison of BB-Relational vs BB-Assocaitions in my app.

[ronen]

I hope you are have set Backbone.Associations.EVENTS_BUBBLE = false. As far as I can remember, BB-rel does not bubble events beyond the immediate level.

I found myself in a quandary...

• with EVENTS_BUBBLE = true, my app would grind to a halt because of the size and interconnectedness of the graph (especially including reverse relations), there'd be zillions of bubbles for every trigger.
• But with EVENTS_BUBBLE = false I didn't have at least the immediate level of event bubbling as per BB-rel, which is something I use heavily. I considered putting together a PR to allow specifying a maximum distance for bubbling, but it seemed too messy -- and besides it's unclear who/how to decide (a property of the relation? an option when triggering?)
• So I went back to using BB-rel for the immediate level of bubbling. But I didn't like having the event store. And I missed the tantalizing possibility of full event paths.

So I pondered how to optimize the event bubbling. And realized the problem was that it was bubbling zillions of triggers that mostly were never listened to. I really only wanted the event to bubble if there was a receiver for it... this led to a proxy-based approach. I originally conceived of it as a PR for BB-Associations, but once I started on it, i realized that by the SoC principle it may as well be a separate plugin. So I made one:

https://github.com/ronen/backbone-chaining

With this plugin, I seem to be able to use BB-Associations with EVENT_BUBBLE = false but still have the full event paths, and everything's running quickly.

BTW In addition to event chaining, it also implements set and get chaining, since they tie in so closely. In that regard, if you find it usable and like the SoC, you could relegate set and get chaining to that plugin.

Note it does not support "nested-change", since I couldn't figure out how to do it in any simple way.

Feedback & suggestions are welcome.

[dhruvaray]

@ronen : Thank you for your contribution and your patience!

I have been caught up last month and my responses haven't been as prompt as I would have liked them to be :(

I agree with you that the optimal way would be to avoid bubbling events if no one was listening to them. As you have pointed out, nested-change is difficult to handle. If you ignore this case, then I have made some minor changes to BB-Associations via an initial commit 631ee35 where event-path end-points (key or parent relation) are recorded during on/off operations. Events are bubbled only if there are non-zero end-points who have expressed interest in those events.

It would be great if you can take a look!

Here is a test case which shows the speed-difference - which can be run locally.

PS : BB-Associations supports fully qualified event paths for custom events too

[dhruvaray]

Here are the fiddle's updated for v0.6.1

Here's a jsfiddle with add + rev-rel (standard bb-rel example) : http://jsperf.com/rev-rel/4
Here's a jsfiddle with set + rev-rel (standard bb-rel example) : http://jsperf.com/rev-rel/5

And the updated documentation page is @ http://dhruvaray.github.io/backbone-associations/speed.html