Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use browserfs/zenfs to overlay one filesystem on top of another #1380

Closed
wants to merge 4 commits into from

Conversation

adamziel
Copy link
Collaborator

@adamziel adamziel commented May 12, 2024

What is this PR doing?

Experiment related to #1379

Uses the zenfs library to create a layered copy-on-write filesystem.

Upsides

  • Enables "mounting" single files by overlaying MemoryFS on top of an underlying NodeFs mounts
  • OverlayFS seems to just work and even implements complex bits like marking parts of the writable tree as deleted even if the deleted paths didn't actually exist in it
  • Not having to go through the complex process of implementing a custom Emscripten Overlay Filesystem

Downsides

  • Introduces additional complexity
  • Zenfs implements Filesystem mechanics from scratch and the semantics might differ from the Emscripten backend in a nuanced way. Also, it's another layer of abstraction which might slow things down. This would require rigorous testing before merging.

Remaining work

  • Use Node.js fs module for zenfs writable FS.
  • Figure out why reading files from Zenfs through Emscripten mount yields a buffer filled with zeros.

Testing instructions

Run this to trigger the code path that mounts ZenFS:

bun packages/playground/cli/src/cli.ts server --skipWordPressSetup

It doesn't do much interesting stuff yet, but at least it mounts the Overlay FS

@adamziel adamziel added [Type] Exploration An exploration that may or may not result in mergable code [Package][@php-wasm] Node labels May 12, 2024
@adamziel
Copy link
Collaborator Author

I'll close this PR – it was a useful exploration we can reuse in the future, but I don't intend on pursuing it now.

@adamziel adamziel closed this May 14, 2024
@james-pre
Copy link

Hi @adamziel, I'm the ZenFS author.

I noticed you are using a custom backend for Emscripten, and seem to be having issues with it. You may be interested in @zenfs/emscripten, which should work correctly with Emscripten. I have not done any in-depth testing, however.

If you do use @zenfs/emscripten, please let me know how it goes. I'd love to get feedback, and using the backend will help me streamline it and fix any bugs.

Hopefully this helps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Package][@php-wasm] Node [Type] Exploration An exploration that may or may not result in mergable code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants