Poor performance on context mutation.

[ysitbon]

Hello,

Happy new year everyone!

We had to do an audit about state a management library for our svelte applications. Performance was mandatory. We started by auditing 3 libraries (more to come) and we did a very quick and dirty perf test where XState registered very low performance when adding 200000 new element to an array.

I'd really wanted to work with XState and push for FSM but since we are working with this kind of huge dataset it seems to be an impossible choice. So I just wanted to share our result with you to know if it was a well known issue and if it is an accepted tradeoff or if not if it's planned for improvement ?

Lib\Iterations	200	2000	20000
immerjs	~ 2.72ms	~ 19.30ms	~ 74.97ms
mobx	~ 17.24ms	~ 81.63ms	~ 490.25ms
xstate	~ 12.94ms	~ 104.51ms	~ 2.13s

Here the repository if you want to look into the code:
https://github.com/bledeabogdan/state-management-audit/tree/main/performance-test

[davidkpiano]

Thanks for the benchmark. There is a plan for improvement: since transitions are deterministic, we can "cache" transitions so that instead of looking through the state node tree to determine what the enabled transitions are, we'd already know them. That way, the only cost is what happens in assign(...).

So the comparison that needs to be made is arr.concat(...) vs. the other techniques, in this case.

Also, you can use immer with XState instead of arr.concat(...); not sure how much that would make a difference though.

Performance is something that we'll tackle when we've finalized the API changes for v5 👍

[ysitbon]

Thanks a lot for your time. I'm even more hyped for v5 now 😄

[ChrisShank]

Just wanted to point out that this benchmark feels like it is not fairly comparing xstate and immer in particular. If you are using a single produce in immer then the equivalent of that with xstate is a single send to the statechart IMO.

[ChrisShank]

I updated the benchmark to reflect this

[ChrisShank]

That said I would love to see a greater emphasis on event throughput in xstate, seems like there is a lot of unexplored territory there.

[ysitbon]

Yes totally agree with you about the fairness 😄. We definitely need to add the vanilla metrics for base comparison too. Thanks a lot for your time and for the additional metrics.