Create Benchmark Suite

[sebmarkbage]

We should create a new benchmark suite for patterns that are relevant to React.

We should write test for four different target audiences:

• JS Engine (Low-level micro-benchmarks of important language features like Object.assign)
• DOM Engine (Low-level micro-benchmarks of document.createElement, innerHTML, cloneNode, updating and setting the .style property etc.)
• React Core Engine (Low-level and real worlds tests of the React algorithm before the DOM layer.)
• Integration Benchmarks (Real world DOM tests that benchmarks the whole thing.)

We need to figure out what format to use for all these benchmarks and how easy they are to incorporate into the engines' benchmark suites.

We should also set up a more stable CI environment to track regressions. The one we have now is too noisy.

[syranide]

👍 "Integration Benchmarks" is incredibly hard to pull off meaningfully in just a single number, I think. I imagine it would be great if each "task" was benchmarked separately, so that trade-offs could be properly considered (animation fluidity/stutter, etc) rather than simply seeing the sum of the most costly operations.

PS. Also letting browsers breathe between benchmarks, too many benchmarks end up benchmarking mostly the GC under enormous pressure rather than something meaningful.

[myocytebd]

Hi, is there some benchmark for "React Core Engine"?
I am building a framework like react-native on v8, and find that "wasted" time is a major bottleneck.
(5-10ms for 20-30 nodes, on 1.9G SandyBridge. And made sure the culprit is not a broken render() implementation or GC)

[gaearon]

@trueadm did some work on creating (4) but we haven't used it much. Browser vendors have worked on tests involving React too.

I don't think it's likely we'll be spending our time on this so I'll close this.