vfs: add --vfs and --vfs-manifest startup flags#3
Draft
pipobscure wants to merge 1 commit into
Draft
Conversation
3 tasks
pipobscure
force-pushed
the
ziparchives
branch
2 times, most recently
from
8eee064 to
0f2b879
Compare
Adds --vfs=<target>, which mounts a directory or ZIP archive and resolves the entry point (process.argv[1]) and all subsequent require()/import resolution against it instead of the real filesystem. A directory target is mounted with RealFSProvider at its own real path, gaining path containment it wouldn't otherwise have; a file target is opened as a read-only ZIP archive (zlib.ZipFile) and mounted with ArchiveProvider, turning that path into a virtual directory. Only paths under the mount are affected - the running program's own node:fs calls to other real paths are untouched, and module resolution never falls back to the real filesystem once it would step outside the mounted target. Getting there requires four of the CJS/ESM loader's module-resolution primitives - package.json reading, nearest-parent/scope lookup, legacy main resolution, and extensionless-file format sniffing - to stop bypassing the public fs module and calling straight into native bindings, since that bypass is exactly what let them ignore the mount. Each gets a VFS-aware replacement that defers to the real native binding unchanged for anything outside an active mount, so behavior for non-mounted paths is identical to before. Native addons are supported: a directory-backed mount dlopens the real underlying file directly; an archive-backed mount extracts the addon to a content-hashed temp file first, since there's no real file to point at. Worker threads inherit an active mount automatically in the common case, and explicitly when the caller supplies its own execArgv that omits it, so sandboxed code can't spawn an "escaped" worker. Also adds --vfs-manifest=<file>, used with a directory --vfs target: the path of every file actually read through the mount - by module resolution or by the program's own node:fs calls - is appended to <file> as it's read, via a small observer hook on the provider base class rather than patching any method. Workers append to the same file directly, since they share the real filesystem with the main thread. Native addons extracted from an archive-backed VFS mount were written under a single shared node-vfs-addons temp directory, keyed only by content hash. Sharing that directory across processes lets one process delete or overwrite a file another process still has dlopen'd/mapped. Scope the directory per pid (node-vfs-addons-<pid>) so each process owns its own extraction cache. Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds --vfs=, which mounts a directory or ZIP archive and resolves the entry point (process.argv[1]) and all subsequent require()/import resolution against it instead of the real filesystem. A directory target is mounted with RealFSProvider at its own real path, gaining path containment it wouldn't otherwise have; a file target is opened as a read-only ZIP archive (zlib.ZipFile) and mounted with ArchiveProvider, turning that path into a virtual directory. Only paths under the mount are affected - the running program's own node:fs calls to other real paths are untouched, and module resolution never falls back to the real filesystem once it would step outside the mounted target.
Getting there requires four of the CJS/ESM loader's module-resolution primitives - package.json reading, nearest-parent/scope lookup, legacy main resolution, and extensionless-file format sniffing - to stop bypassing the public fs module and calling straight into native bindings, since that bypass is exactly what let them ignore the mount. Each gets a VFS-aware replacement that defers to the real native binding unchanged for anything outside an active mount, so behavior for non-mounted paths is identical to before.
Native addons are supported: a directory-backed mount dlopens the real underlying file directly; an archive-backed mount extracts the addon to a content-hashed temp file first, since there's no real file to point at. Worker threads inherit an active mount automatically in the common case, and explicitly when the caller supplies its own execArgv that omits it, so sandboxed code can't spawn an "escaped" worker.
Also adds --vfs-manifest=, used with a directory --vfs target: the path of every file actually read through the mount - by module resolution or by the program's own node:fs calls - is appended to as it's read, via a small observer hook on the provider base class rather than patching any method. Workers append to the same file directly, since they share the real filesystem with the main thread.