JS performance: operators don't get inlined

[frank-rollpin]

We are developing a computationally intensive web app, and one of the performance-critical parts is dealing with large bit sets. The program works fast enough in DartVM, however there is about 6x performance degradation compared to running the JS output in regular Chrome.

After running micro-benchmarks (I was careful, and the results are consistent across runs - see the attached code), we found out few interesting patterns. The two tests set every other bit in a bitset to true, the "inline" test is doing it in a straightforward manner, while the "operator" test sets it via the defined operator. "Chrome (min)" is the attached benchmark; "Chrome (app) is testing absolutely the same code, the only difference is that it is compiled as part of our big application. Here are the results (Win7 / DartVM 1.19, Chromium 1.19, Chrome 52):

	DartVM	Chromium	Chrome (min)	Chrome (app)
inline	1092	2373	2450	3080
operator	990	2423	3277	6055

... and the code (web.zip):

class BitSetPerfTest {
  static const int size = 100000;
  static const int repeats = 100;
  Uint32List _data = new Uint32List(size);

  bool operator [] (int pos) => ((_data[pos >> 5] & (1 << (pos & 0x1f))) != 0);

  void operator []= (int pos, bool value) {
    if (value)
      _data[pos >> 5] |= 1 << (pos & 0x1f);
    else
      _data[pos >> 5] &= ~(1 << (pos & 0x1f));
  }

  void testSetOperator() {
    print('test set operator');
    for (int n = 0; n < repeats; n++)
      for (int i = 0; i < size * 32; i++)
        this[i] = i % 2 == 0;
  }

  void testInlinedSet() {
    print('test inlined set');
    for (int n = 0; n < repeats; n++)
      for (int i = 0; i < _data.length * 32; i++)
        if (i % 2 == 0)
          _data[i >> 5] |= 1 << (i & 0x1f);
        else
          _data[i ~/ 0x20] &= ~(1 << (i & 0x1f));
  }
}

Upon further investigation, we found that the '=' operator got inlined in the benchmark code, but did not get inlined in the real application, which likely resulted in 2x performance penalty:

------------------ Chrome (min) : the operator is inlined

P.G("test set operator")
for(z=this.a,y=0;y<100;++y)for(x=0;x<32e5;++x**){w=x>>>5**
v=x&31
if(C.a.A(x,2)===0){if(w>=1e5)return H.m(z,w)

------------------ Chrome (app) : there is no inlining

P.aJ("test set operator")
for(z=0;z<100;++z)for(y=0;y<32e5;++y**)this.i(0,y,C.c.ax(y,2)===0)**},"$0","ga23",0,0,3],
a45:[function(){var z,y,x,w,v

Few questions (I'm not sure if this post qualifies as a bug report, but hopefully it gives some food for thoughts):

• Why is DartVM much faster than the one in Chromium?
• What defines whether a method gets inlined or not?
• Are there any techniques we can use to help compiler produce better code?
• Are there any tools that would help us automate cross-platform performance tests?

[harryterkelsen]

@rakudrama what do you think?

[rakudrama]

The dart2js compiler won't inline things that look like they will cause bloated downloads.
The heuristics are subject to change, but depend on the size of the method, whether there is only one call to the method, whether the call is in a loop etc.
There are lots of things that prevent inlining, like try-catch and methods with two returns.

Try making the method as tiny as possible (in tokens, meaningful identifiers are OK).

  void operator []= (int pos, bool value) {
    int index = pos >> 5, mask = 1 << (pos & 0x1f);
    if (value)
      _data[index] |= mask;
    else
      _data[index] &= ~mask;
  }

Write more realistic benchmarks. It is unlikely your datastructure is called BitSetPerfTest. You probably have a BitSet, use that and exercise it.

It is important for code quality that dart2js can see that []= is always passed an int and a bool.
Don't use mirrors. Using mirrors means that dart2js will assume that you call []= via reflection with unknown values.
You can use lots of types and --trust-type-annotations, but be sure the program runs properly in checked mode.

As the program gets larger, it is more likely that it contains a[b] = c where it is impossible to tell what a is, e.g. a came from an untyped Map. dart2js has to assume a could be one of your BitSets, and it has to deal with whatever b is. It is allowed to be null, in which case the code will contain checks against null values.
Since the program contains lots of stuff that has operator[]= that is hard to analyse, use a different name to reduce the possibility of confusion, e.g. getBit and setBit.

Use final fields.

Everything in my talk at last year's dart summit still applies to dart2js performance.