benchmarks affect each other?

[leeoniya]

hey @dfilatov

i was running the SSR benchmarks and noticed a significant difference when running each independently vs all at once. i assume there's some mem leaks or poor GC interactions which affect the results between bench runs.

this is what i'm seeing: https://github.com/leeoniya/domvm/tree/3.x-dev/demos/bench/ssr

[dfilatov]

Hi, @leeoniya
Actually, when I was implementing it, I had only noticed that Vue affected all other results after it. So I had to put it last.
Currently I've tried to run them separately and haven't noticed any significant difference on my macbook.

[leeoniya]

k, another point to consider is running them on a desktop/macbook vs a linux server (where SSR will almost certainly be running). i could not get vidom metrics to be faster than inferno on a 4096 Linode instance in node v8.1.3 or on my win10x64 desktop.

just thought i'd point this out. you may also be interested in the recently started https://github.com/ivijs/ssr-benchmark which includes vidom's benchmarks.

original thread here: localvoid/ivi#1

on linode domvm does about 20x React's perf (not 14x as mentioned in the thread, which was my desktop metrics)

[dfilatov]

Yes, the difference between vidom and inferno is tiny on nodejs 8.x, about 10% on my macbook, unlike nodejs 6.x where it can reach 40-50%.