Polyfill around typed arrays

[kripken]

https://github.com/mozilla/pdf.js/blob/master/web/compatibility.js#L7-49

is a polyfill for some of the typed arrays spec, letting older browsers use the typed arrays API (slowly).

Would be nice for emscripten to use that too, so typed array builds would work and not fail as they do now, if the environment lacks typed arrays. We do need more support than in that link, though (different size arrays, shared buffers, etc.).

[pskupinski]

So I was thinking about this problem and because it may not be possible to have a decently performing polyfill that perfectly conforms to native typed arrays I have thought up a decent alternative as outlined below.

So our biggest problem in having a polyfill that acts exactly like native typed arrays is that such as in the below example:

var a = new Int8Array(1);
a[0] = 0xff;  // need a setter to have the line below not print 255
console.log(a[0]); // -1

Having a setter(such as with __defineSetter__) for every possible index would be far too slow and would not work in many older browsers that would be nice to target. If we have a function like setIndex which would take an index and a value (I went with setIndex as set is taken and does not perform this functionality) and used that for setting values at the specified index a javascript implementation would be possible and it could work with the native typed arrays by simply adding that function to them. The hard work of making sure the value properly fits the type of the array would be done in this function(such as turning 255 into -1 with Int8Array in the above example) for browsers that would need a typed arrays implementation and then accessing by index could be done as normal.

So for the above code example it would become this:

var a = new Int8Array(1);
a.setIndex(0, 0xff);
console.log(a[0]); // -1

I would love to hear your thoughts on this idea.

[kripken]

Yeah, that makes sense, I agree. However, moving from a[0] = X to a.setIndex(0, X) would make us much slower if there are typed arrays. But there might be a solution: We can check if typed arrays are present, and if not, then rewrite the script then and there to replace all a[0] = X with a.setIndex(0, X) and so forth.

So basically we would compile code for typed arrays, and there would be a 'wrapper' script that checks for typed arrays, then if necessary rewrites the script, then eval's it.

The rewriting could be done with an AST parser like UgilifyJS, but probably this is simple enough that some regexps can handle it, I hope.

[kripken]

To clarify, what makes this possible is that we know the names of our typed arrays (HEAP8, HEAP16, etc.), and nothing conflicts with them. If we didn't, this would in general be impossible.

[pskupinski]

Yes, it would definitely be slower to have to call a function on top of array access for situations where there is native support for typed arrays and doing some sort of rewriting to work around that sounds like a great idea. I'll take more of an in-depth look into the Emscripten internals so I can better understand how this would fit into the system.

I'll work on implementing this as I'm pretty excited to see this work.

[kripken]

Btw, one complication I just thought of is that parsing the output in -O2 and -O3 will be much harder, since closure compiler is run on it, which renames all the variables. It's still possible, but trickier. Might make sense to ignore that case at first.

[kripken]

Polyfilling this is hard, and becoming quickly less important as all browsers get typed arrays, so closing this.

[niutech]

@kripken And what about IE9, which has 21% market share? Can't you just use a shim, which uses both __defineSetter__ and defineProperty?

[kripken]

It's worth trying that shim you linked to. But I suspect it will be so much slower that it isn't worth it. Let's see numbers though.

[niutech]

@kripken Thank you!
The slowness in IE could be an impulse to replace that browser with a faster one.

[kripken]

Regarding numbers, someone with IE9 needs to get them (I don't have a windows machine myself).

[niutech]

I have IE9, which exactly numbers should I get?

[kripken]

I would be curious to take a benchmark, say tests/memops.cpp, and compile that with -O2 --closure 0. Then run it in Firefox, Chrome and IE10 on your machine. Then add that polyfill stuff for typed arrays so that it can work in IE9, and compare the speed there to the previous results.

[hhm0]

This would be really nice for the Wii browser,and for similar "embedded" browsers, which cannot always be updated.

That way, code which is already online, and is small/fast enough to run on these browsers (this probably would, for example), can be run out-of-the-box.

[kripken]

Typed arrays are everywhere at this point, and we depend on them so heavily, I don't think we could polyfill around them in a useful way.