Another benchmark

[scriby]

I've been doing some experimenting with immer. I really like how much simpler it is to write reducers with it!

I was doing some benchmarking vs ImmutableJS and wanted to check if my results make sense.

The benchmark looks like this:

• Trigger a single action which is handled by one reducer
• The reducer produces a new state by appending the item's id to the end of an array and adding the item into a map (or plain JS object for immer)

I'm triggering the action 3,000 times, and then measuring the performance of triggering the action another 1,000 times.

The immutable JS reducer takes about 400ms, and the immer reducer takes about 4 seconds. Here's the immer profile:

It looks like the main cost is incurred by producing copies of the state as the state is supposed to be immutable, which has to happen for each triggered action. And that's also the main part that ImmutableJS is able to optimize away.

I just wanted to check with you on whether that basically made sense and whether you had any thoughts on it. (When running immer, proxies are used and freezing is off. Browser is Chrome.)

Thanks,

Chris

[mweststrate]

Just checking: Did you test with useProxies true and autoFreeze false? Can you make a PR out of this benchmark? Op za 7 apr. 2018 02:42 schreef scriby <notifications@github.com>:

…

I've been doing some experimenting with immer. I really like how much simpler it is to write reducers with it! I was doing some benchmarking vs ImmutableJS and wanted to check if my results make sense. The benchmark looks like this: - Trigger a single action which is handled by one reducer - The reducer produces a new state by appending the item's id to the end of an array and adding the item into a map (or plain JS object for immer) I'm triggering the action 3,000 times, and then measuring the performance of triggering the action another 1,000 times. The immutable JS reducer takes about 400ms, and the immer reducer takes about 4 seconds. Here's the immer profile: [image: image] <https://user-images.githubusercontent.com/815188/38449255-2b6b0be2-39c1-11e8-9d51-7c8ed0928b87.png> It looks like the main cost is incurred by producing copies of the state as the state is supposed to be immutable, which has to happen for each triggered action. And that's also the main part that ImmutableJS is able to optimize away. I just wanted to check with you on whether that basically made sense and whether you had any thoughts on it. Thanks, Chris — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#138>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABvGhJLHkTxK9Ng4g-EbxfRgTAasOwvRks5tmAtrgaJpZM4TK3Np> .

[scriby]

Yeah freeze was off and proxies on. I’ll send a PR for the benchmark Monday. Thanks!

…

On Apr 6, 2018, at 11:34 PM, Michel Weststrate ***@***.***> wrote: Just checking: Did you test with useProxies true and autoFreeze false? Can you make a PR out of this benchmark? Op za 7 apr. 2018 02:42 schreef scriby ***@***.***>: > I've been doing some experimenting with immer. I really like how much > simpler it is to write reducers with it! > > I was doing some benchmarking vs ImmutableJS and wanted to check if my > results make sense. > > The benchmark looks like this: > > - Trigger a single action which is handled by one reducer > - The reducer produces a new state by appending the item's id to the > end of an array and adding the item into a map (or plain JS object for > immer) > > I'm triggering the action 3,000 times, and then measuring the performance > of triggering the action another 1,000 times. > > The immutable JS reducer takes about 400ms, and the immer reducer takes > about 4 seconds. Here's the immer profile: > > [image: image] > <https://user-images.githubusercontent.com/815188/38449255-2b6b0be2-39c1-11e8-9d51-7c8ed0928b87.png> > > It looks like the main cost is incurred by producing copies of the state > as the state is supposed to be immutable, which has to happen for each > triggered action. And that's also the main part that ImmutableJS is able to > optimize away. > > I just wanted to check with you on whether that basically made sense and > whether you had any thoughts on it. > > Thanks, > > Chris > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#138>, or mute the thread > <https://github.com/notifications/unsubscribe-auth/ABvGhJLHkTxK9Ng4g-EbxfRgTAasOwvRks5tmAtrgaJpZM4TK3Np> > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[scriby]

PR is here: #140

[wooandoo]

hi,

immer seems promising.

for information, I did another benchmark: immer vs ramda lens vs shallow copy:

• https://jsbench.me/2mjjchh1pm/1
• https://measurethat.net/Benchmarks/Show/3379/0/immer-setautofreezefalse-vs-shallow-vs-ramda-lens

same benchmarks, the first one has to be started several times to work, but its results are closest to those I get on NodeJS

with a existing "database" of 100 items (by default):

• immer is 2 times less performant (28,144 ops/s)
• ramda lens ~ shallow copy (40,498 ops/s )
• shallow copy gets the best results (43,369 ops/s)

with a existing "database" of 1000 items:

• immer is relatively better (3,625 ops/s)
• shallow copy is second (4,791 ops/s)
• ramda lens is the better (5,587 ops/s)

with a existing "database" of 10000 items:

• immer (346 ops/s)
• shallow copy (417 ops/s)
• ramda lens (575 ops/s)

with a existing "database" of 10 items:

• immer (98,625 ops/s)
• ramda lens (118,113 ops/s)
• shallow copy (295,710 ops/s)

the performance varies a lot !
but, in a Redux use, the state has between 10 to 100 items per data type

[mweststrate]

Just merged #140.

As a reminder, note that the test is kinda artifical; if you need to add X-thousand items that should be done in a single produce call! If this is part of an async process (e.g. new data coming in over a websocket), then you will have the overhead of the individual produce calls; however, then these times will be typically be neglect-able compared to the duration of the websocket handling, http traffic and probably all the other things the app is doing with your data :-).