Revisit built-in support for TextMate grammars

[bolinfest]

Maintaining both a TextMate grammar for VS Code as well as a Monarch grammar is a lot of work. As a result, the Monarch grammar often lags (e.g., lack of Python 3 support) or is nonexistent (Hack). In the absence of an official TextMate-to-Monarch converter, bringing TextMate support to Monaco seems like the best solution.

The justification for requiring the use of Monarch grammars instead of TextMate grammars on the README seems to cite some outdated statistics. For example, it notes:

We have experimented with Emscripten to compile the C library to asm.js, but performance was very poor even in Firefox (10x slower) and extremely poor in Chrome (100x slower).

Looking at old issues, I see these stats date back to at least 2016:

#171

The README also notes:

We can revisit this once WebAssembly gets traction in the major browsers, but we will still need to consider the browser matrix we support, i.e. if we support IE11 and only Edge will add WebAssembly support, what will the experience be in IE11, etc.

Looking at the latest data, WASM appears to have fairly broad support today, though IE11 is admittedly a notable exception:

https://caniuse.com/#feat=wasm

In terms of implementing support for TextMate grammars, it appears that there are two possible paths for filling in the missing oniguruma functionality:

• A "pure JS" polyfill like onigurumajs, as suggested on TextMate - OnigurumaJS #722.
• A WASM implementation. https://github.com/NeekSandhu/onigasm claims to be 2x slower than the native V8 implementation, which I think many would choose over trying to maintain two grammars.

I think it would be ideal to get something working and then give the client the option to specify either a TM grammar or a Monarch one. (Or if we go the WASM route, specify the preferred TM grammar and the Monarch one as a fallback in browsers that either do not support WASM / lack performant support for WASM.)

I am happy to help, but I would like to know what the team is open to so that I don't work on something that has no chance of being upstreamed.

[rcjsuen]

@bolinfest Also see #1915.

[bolinfest]

@rcjsuen Whoops, thanks! Closing as a dupe of #1915.