Playground not working with "Uncaught TypeError: tree is not defined"

[ahelwer]

I've been trying to get the playground to work with my grammar but when the web UI pops up in Firefox or Chrome I just get this error and nothing works:

RuntimeError: abort(Assertion failed: bad export type for `_ZTVN10__cxxabiv117__class_type_infoE`: undefined). Build with -s ASSERTIONS=1 for more info.
    ee http://127.0.0.1:8000/tree-sitter.js:1
    S http://127.0.0.1:8000/tree-sitter.js:1
    be http://127.0.0.1:8000/tree-sitter.js:1
    M http://127.0.0.1:8000/tree-sitter.js:1
    p http://127.0.0.1:8000/tree-sitter.js:1

If I type then I get this error over & over:

Uncaught TypeError: tree is undefined
    handleCursorMovement http://127.0.0.1:8000/playground.js:326
    later http://127.0.0.1:8000/playground.js:453
    setTimeout handler*debounce/< http://127.0.0.1:8000/playground.js:457
    Gn https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    Gn https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    Gn https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    Xn https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    poll https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    init https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1

Am I doing something obviously wrong?

Version Info:

• emcc 2.0.16
• tree-sitter-cli 0.20.0
• Windows 10
• Firefox 90.0.2
• Chrome 92.0.4515.107

[ahelwer]

I've tried setting up a playground with a very simple test grammar, and it works, so it's something unique to my grammar which is causing the issue.

[ahelwer]

Even if I make my top-level rule something simple, like source_file: $ => /\d+/, the same failure occurs. From looking at the generated parser.c and node-types.json tree-sitter will still use rules even if they aren't reachable, so it must have to do with that.

[ahelwer]

I've isolated the issue to my external scanner. If I remove all external scanner symbols from my grammar and move/delete the scanner.cc file out of the src directory, then the playground starts up and works without error. I can get a playground working with a simple test grammar + small external scanner, so there's something unique to this grammar's external scanner that is messing things up.

[maxbrunsfeld]

Yeah, there are some limitations to how dynamic linking works with Emscripten, so the core library (web-tree-sitter) has to try to include all of the C/C++ standard library symbols that external scanners will need. Your scanner appears to use a symbol from libc++ that isn't included. I think it has to do with your use of std::function. But that's just a guess.

[ahelwer]

Thanks @maxbrunsfeld, removing the dependency on <functional> fixed the issue!

[ahelwer]

Actually it does work to start, but stops at a certain point. I get this error now:

Uncaught (in promise) TypeError: n is undefined
    t http://127.0.0.1:8000/tree-sitter.js:1
    parse http://127.0.0.1:8000/tree-sitter.js:1
    handleCodeChange http://127.0.0.1:8000/playground.js:115
    it https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    jn https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    Gn https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    Gn https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    Gn https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    Gn https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1
    Yn https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.45.0/codemirror.min.js:1

My changes are in this branch; to repro:

• Clone
• npm install
• tree-sitter build-wasm (this takes a couple of minutes)
• tree-sitter playground
• In the window, add the following text (which shouldn't trigger any errors yet):

  ---- MODULE Test ---
  ====
  

• Open your browser dev console, start typing, then see the errors:

  ---- MODULE Test ---
  op == x
  ====
  

[ahelwer]

For ease of repro, I've now set up a github page with the playground: https://tlaplus-community.github.io/tree-sitter-tlaplus/
Open your browser then type anything in between the ---- and ==== to see the error. Through testing I've also isolated this to, again, the external scanner.

[ahelwer]

All right I completely rewrote how my lexer works in my external scanner (now looks very similar to the auto-generated lexer) and somehow that fixed the issue. I don't know why. Maybe the old lexer used some newer C++ features that didn't play well with emscripten. You can see the changes to scanner.cc in this commit.