Activate DuckDB worker lazily - #14938
Merged
Merged
Conversation
|
E2E Tests 🚀 Why these tags?
More on automatic tags from changed files. |
Contributor
softwarenerd
approved these changes
Jul 17, 2026
softwarenerd
left a comment
Contributor
There was a problem hiding this comment.
Checks out in testing. Approved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



This change reduces Positron's steady state memory usage by 50-100MB, unless you have Data Explorer tabs open against CSV files (or others that use DuckDB).
The
positron-duckdbextension activatedonStartupFinishedand forked its native DuckDB worker (almost 100MB!) inactivate(), so the worker was resident in every window even when no data file was ever opened. Here's Positron's Process Explorer on a fresh boot of a recent release build:Note that the
duckdbWorker.jsworker is loaded and consuming 98MB:This was especially wasteful in web/server builds where the extensions are a greater fraction of total memory use. This PR makes both the extension and its worker lazy, and reclaims the worker's memory when it's no longer in use.
Summary
onPositronDataExplorerBackend:positron-duckdbevent instead ofonStartupFinished.MainThreadDataExplorerfires this event (viaactivateByEvent) the first time a dataset routed to the provider is accessed, so the extension stays dormant until a data file is actually opened. The existing 30s provider-registration wait inExtHostDataExplorer.$handleRpccovers the activation window.DuckDBInstance.create()no longer forks the worker. The child process is spawned lazily on the first query (the path already used for crash recovery), i.e. only when a dataset is opened.disposeBackendlifecycle signal (main thread -> ext host -> the provider's optionalcloseDatasethandler) delivers the close notification without activating a dormant extension.Release Notes
New Features
Bug Fixes
Validation Steps
@:data-explorer @:duck-db @:connections
duckdbWorker.jsnode process is running and thepositron-duckdbextension is not activated..csv,.parquet, or.duckdb) in the Data Explorer. The extension activates and the worker process spawns; the data renders normally.Existing e2e coverage exercises the file-open/connection path that lazy activation must not break; new unit tests cover the worker lifecycle (lazy spawn, idle shutdown, respawn, cancelled shutdown) and the
disposeBackendclose signal.