Skip to content

Commit

Permalink
Fix JSON integer parsing
Browse files Browse the repository at this point in the history
  • Loading branch information
RedCMD committed Mar 7, 2024
1 parent 5b30aff commit c17dbf8
Show file tree
Hide file tree
Showing 7 changed files with 8,345 additions and 8,382 deletions.
Binary file modified out/tree-sitter-jsontm.wasm
Binary file not shown.
84 changes: 43 additions & 41 deletions src/tree-sitter/README.md
Original file line number Diff line number Diff line change
@@ -1,48 +1,50 @@
Tree-sitter -> wasm

https://emscripten.org/docs/getting_started/downloads.html
https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/README.md
https://tree-sitter.github.io/tree-sitter/creating-parsers


install python with optional `py` launcher and `PATH` option

install emscripten in a separate private folder
`git clone https://github.com/emscripten-core/emsdk.git`
`cd emsdk`
`git pull`
(the latest version (`3.X.X`) doesn't seem to work with the `vscode-parse-tree` extension on Windows. Install version `2.0.34` instead)
`./emsdk install 2.0.34`
`./emsdk activate 2.0.34 --permanent`
`./emsdk_env.bat`
restart powershell and/or vscode

install tree-sitter in the extension folder
cd `/out/tree-sitter/`
`npm install tree-sitter-cli`

compile wasm
run `build_grammar.ps1`


to install tree-sitter in your own personal extension
`mkdir tree-sitter-your_language`
`cd tree-sitter-your_language`
`npm init` (creates `package.json`)
`npm install --save nan`
`npm install --save-dev tree-sitter-cli`
add `./node_modules/.bin` to `PATH`
`grammar.js` => ```js
Tree-sitter -> wasm

https://emscripten.org/docs/getting_started/downloads.html
https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/README.md
https://tree-sitter.github.io/tree-sitter/creating-parsers


install python with optional `py` launcher and `PATH` option

install emscripten in a separate private folder
`git clone https://github.com/emscripten-core/emsdk.git`
`cd emsdk`
`git pull`
(the latest version (`3.X.X`) doesn't seem to work with the `vscode-parse-tree` extension on Windows. Install version `2.0.34` instead)
`./emsdk install 2.0.34`
`./emsdk activate 2.0.34 --permanent`
`./emsdk_env.bat`
restart powershell and/or vscode

install tree-sitter in the extension folder
cd `/out/tree-sitter/`
`npm install tree-sitter-cli`

compile wasm
run `build_grammar.ps1`


to install tree-sitter in your own personal extension
`mkdir tree-sitter-your_language`
`cd tree-sitter-your_language`
`npm init` (creates `package.json`)
`npm install --save nan`
`npm install --save-dev tree-sitter-cli`
add `./node_modules/.bin` to `PATH`
`grammar.js` =>
```js
module.exports = grammar({
name: 'your_language',

rules: {
// TODO: add the actual grammar rules
source_file: $ => 'hello'
}
});```
`tree-sitter generate`
`example-file` => `hello` (UTF8 encoding!!)
`tree-sitter parse example-file`

https://raw.githubusercontent.com/tree-sitter/tree-sitter/master/cli/src/generate/grammar-schema.json
});
```
`tree-sitter generate`
`example-file` => `hello` (UTF8 encoding!!)
`tree-sitter parse example-file`

https://raw.githubusercontent.com/tree-sitter/tree-sitter/master/cli/src/generate/grammar-schema.json
2 changes: 1 addition & 1 deletion src/tree-sitter/tree-sitter-json/grammar.js
Original file line number Diff line number Diff line change
Expand Up @@ -463,7 +463,7 @@ module.exports = grammar({
"false",
),
null: $ => "null",
integer: $ => /\d+/,
integer: $ => /-?(0|[1-9]\d*)(\.\d+)?([eE][+-]?\d+)?/,
_string: $ => token(
repeat1(
choice(
Expand Down
2 changes: 1 addition & 1 deletion src/tree-sitter/tree-sitter-json/src/grammar.json
Original file line number Diff line number Diff line change
Expand Up @@ -4305,7 +4305,7 @@
},
"integer": {
"type": "PATTERN",
"value": "\\d+"
"value": "-?(0|[1-9]\\d*)(\\.\\d+)?([eE][+-]?\\d+)?"
},
"_string": {
"type": "TOKEN",
Expand Down
9 changes: 4 additions & 5 deletions src/tree-sitter/tree-sitter-json/src/node-types.json
Original file line number Diff line number Diff line change
Expand Up @@ -747,11 +747,6 @@
]
}
},
{
"type": "integer",
"named": true,
"fields": {}
},
{
"type": "item",
"named": true,
Expand Down Expand Up @@ -1551,6 +1546,10 @@
"type": "false",
"named": false
},
{
"type": "integer",
"named": true
},
{
"type": "key",
"named": true
Expand Down

0 comments on commit c17dbf8

Please sign in to comment.