perf: async tokenization with worker threads #465
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ThomasK33
force-pushed
the
thomask33/10-28-add_tokenizer_pool
branch
15 times, most recently
from
ed3632f to
9055fe1
Compare
ThomasK33
changed the title
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ThomasK33
force-pushed
the
thomask33/10-28-add_tokenizer_pool
branch
from
9055fe1 to
f997ff8
Compare
|
@codex review |
1 similar comment
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. Nice work! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Tokenization now runs in worker threads to prevent blocking during heavy encoding operations, maintaining UI responsiveness. Previous synchronous approach with fallback approximations caused UI lag when calculating token counts for large chat histories. Moving tokenization off the main thread keeps the UI responsive at 60fps even during expensive tokenization workloads. Key changes: - Worker thread pool using node:worker_threads - All tokenizer APIs converted to async - IPC channels for renderer-side tokenization - Frontend LRU cache with Suspense integration - Real-time token estimates in ChatInput - Updated ~50 test files for async API
ThomasK33
force-pushed
the
thomask33/10-28-add_tokenizer_pool
branch
from
cd26335 to
5c7d7a0
Compare
1 participant
Tokenization now runs in worker threads to prevent blocking during heavy encoding operations, maintaining UI responsiveness.
The previous synchronous approach with fallback approximations caused UI lag when calculating token counts for large chat histories.
Moving tokenization off the main thread keeps the UI responsive even during expensive tokenization workloads.
Key changes: