WasmFS Documentation

[tlively]

WasmFS should be well documented, with docs covering this non-exhaustive list of topics:

• How to build with WasmFS
• How to configure various backends and reason about when caches, etc. would be helpful.
• How to write your own backend.

[curiousdannii]

My project has two things that I'd like to use WasmFS for, so it would be great if the docs explained both of these:

1. A custom async FS. I have a custom FS currently, but it's hampered by not being async
2. Async stdin, which I currently implement by wrapping getc with a function that calls (if it's getting stdin) an EM_ASYNC_JS function. I'm lucky that the project I'm porting only accesses stdin via getc, as it would be messy to have to wrap lots of other functions too.

Also, enabling and mounting FSes used to happen in JS code. Will that still be the case? Or will we be able to put this code into the wasm. My port doesn't modify main, so it would be nice if this doesn't require that either...

this.EFS = new EmglkenFS(this)
Module.FS.mkdir('/emglken')
Module.FS.mount(this.EFS, {}, '/emglken')
Module.FS.chdir('/emglken')

[tlively]

re: mounting, I expect we will have both C and JS APIs for that. The story for Async APIs hasn't quite been figured out, but that's tracked in #15964.

[curiousdannii]

@tlively #15964 is about a new async API, right? I was expecting that it would possible to make the standard posix FS API async with Asyncify.

[tlively]

Oh gotcha. Yes, we might have to do something to get Asyncify working as expected as well. (Or maybe it will Just Work, not sure). I filed #16053 to track that separately.

[nachoab]

Is there some example of compiling including OPFS? Setting WASMFS=1 doesn't seem to include it, even though it looks like it is. Also, using current FS you have to use the mount method to specify a different FS provider, does WasmFS use this same API?

[mere-human]

Is there some example of compiling including OPFS?

@nachoab I think the best example would be tests, for instance https://github.com/emscripten-core/emscripten/blob/main/test/wasmfs/wasmfs_opfs.c

[curiousdannii]

It's taken a long while, but I'm ready to start working on my own WasmFS backend.

How "ready" is WasmFS now? I know some people are using it, but I also see lots of open issues and tickets in the project/milestone. It seems like there's still not really any docs, we just have to learn from the other implementations?

I suspect it might not be that difficult to use WasmFS with Asyncify, because you can put anything through Asyncify. The question is more about what's efficient and ergonomic.

An alternative idea I had was to somehow replace stdio.h with a custom implementation, but that would probably be much more complex.

[tlively]

We have some very large applications using WasmFS, especially with the OPFS backend, so it's "ready" for production use in the sense that it is stable and correct enough. We haven't been able to implement as many new features as we wanted, though, such as other concrete backends (for IndexedDB especially) and virtual backends for caching and layering, etc. We also haven't been able to finish all the compatibility work that we want to have done to allow us to turn on WasmFS by default.

[art926]

Could you update
https://emscripten.org/docs/api_reference/Filesystem-API.html#file-systems
to mention OPFS, please?) We're very interested in using it.