Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add telemtry for how long it takes to parse files with tree-sitter #213565

Merged
merged 43 commits into from
Jul 23, 2024

Conversation

alexr00
Copy link
Member

@alexr00 alexr00 commented May 27, 2024

Part of #210475

@alexr00 alexr00 self-assigned this May 27, 2024
@alexr00 alexr00 enabled auto-merge (squash) May 27, 2024 14:26
@alexr00 alexr00 requested a review from hediet May 27, 2024 14:27
@vscodenpa vscodenpa added this to the May 2024 milestone May 27, 2024
@alexr00 alexr00 force-pushed the alexr00/treeSitterParserService branch from 42c4efa to 7f0678a Compare May 27, 2024 14:48
@hediet
Copy link
Member

hediet commented May 27, 2024

CI failure

Copy link
Member

@hediet hediet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!
There are still some race conditions that I think should be fixed before merging this though.

Copy link
Member Author

@alexr00 alexr00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hediet, thanks for the review! I tried to capture all of your feedback.

@alexr00 alexr00 requested a review from hediet July 18, 2024 10:09
}

override dispose() {
super.dispose();
this._treeSitterTree?.dispose();
this._unregisterModelListeners();
this._languageSessionDisposables.dispose();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not needed, as the this._register call takes care of that :)
The nice thing about _register is that you can see at the creation location in the source that it is eventually going to be disposed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦‍♀️

@alexr00 alexr00 requested a review from hediet July 18, 2024 13:17
}
return this._onDidChangeContent(model, e);
}).catch((e) => {
this._logService.error('Error parsing tree-sitter tree', e);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think onUnexpectedError might also be good here, because then you'd see it in telemetry.

try {
tree = this.parser.parse((index: number, position?: Parser.Point) => this._parseCallback(model, index), this.tree);
} catch (e) {
// parsing can fail when the timeout is reached, will resume upon next loop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you check the error type and re-throw the error for non-expected errors?

@alexr00 alexr00 merged commit 6c15586 into main Jul 23, 2024
6 checks passed
@alexr00 alexr00 deleted the alexr00/treeSitterParserService branch July 23, 2024 12:59
@alexr00 alexr00 restored the alexr00/treeSitterParserService branch July 24, 2024 08:14
nagi1999a pushed a commit to nagi1999a/vscode that referenced this pull request Jul 26, 2024
…icrosoft#213565)

* Make space for tree sitter

* Add the tree sitter wasm file

* Very naive tree-sitter syntax highlighting for html, with a layer breaker

* Update tree when content changes

* WIP for making abstract tokens class

* Handle theme changes

* Replace entire text model value with parse callback

* Perf improvements

* Add tree-sitter-typescript

* Add typescript + better initial parsing

* Refactor into tree parsing service and fix flaw in parse callback

* Remove things that aren't the parser service

* Add yielding

* Remove changes that aren't required for PR

* Remove more file changes

* Reduce yield to 50 ms

* Fix incremental parsing

* Try update node-abi

* Revert "Try update node-abi"

This reverts commit df28801.

* Update text buffer chunk api

* fix build

* Remove tree-sitter dependency

* Adopt new, as yet unpublished, `@vscode/tree-sitter-wasm` package

* Use published `@vscode/tree-sitter-wasm` package

* Break `TreeSitterTree` and `TreeSitterParserService` into better pieces
and:
- document the order of editor changes
- use service injection where `TextModel` is constructed

* Fix tests

* Remove unneeded import

* Fix missing tree-sitter-wasm in web and remote

* Make package.jsons match

* Add @vscode/tree-sitter-wasm to web loader config

* Try using importAMDNodeModule

* PR feedback

* Add race condition test for changing language while loading language

* Use same timeout

* Queue content changes

* Remove override dispose

* Move queue into TreeSitterTree

---------

Co-authored-by: Peng Lyu <penn.lv@gmail.com>
@alexr00 alexr00 deleted the alexr00/treeSitterParserService branch July 29, 2024 09:31
@vs-code-engineering vs-code-engineering bot locked and limited conversation to collaborators Sep 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants