User selected file accessible for WebAssembly code

[hsgarciacaballero]

Hi everyone! I have been looking for information on how WebAssembly (web environment) could benefit from using files that have been previously selected by the user, but I have not found a proper solution. This is why I am opening this issue to discuss a possible proposal. The use case I am aiming for would be something as follows:

We have a library written in C that is able to parse large medical files and we would like to reuse the logic of this library on the web for performance reasons. WebAssembly seems to be a perfect candidate to do so. The C library relies on standard C functions such as fopen to open and read the structure of the medical file. Reading the medical file this way is quite efficient as the library does not have to read the entire file but just the header. Additionally, the user can retrieve more data from the file on demand.

I guess this use case is pretty common for many other libraries independently of the language they are written in.

One possible way to address this use case would be by means of the Virtual File System, and copying the entire content of the selected file into it. Although this approach works out, it has some drawbacks. For example, the file has to be entirely copied into memory. For smaller files it would not represent a problem. However, bigger ones would definitively be an issue.

An alternative solution could be that the Javascript engine "flags" the file selected by the user as "readable" for the WebAssembly program being executed exposing a sort of "fake path". This way, libraries would not require to be modified (i.e., they would continue reading files provided as paths), files would not have to be copied into the virtual file system and it would still be safe as the code executed cannot read any random file from user's computer but only those the user has selected manually.

[binji]

It sounds like emscripten's filesystem support may be able to help here. In particular, IDBFS and WORKERFS seems like they may be a good fit:

This file system provides read-only access to File and Blob objects inside a worker without copying the entire data into memory and can potentially be used for huge files.

Maybe @kripken can give you some other suggestions.

[kripken]

Yeah, WORKERFS is basically intended for exactly this, it can avoid loading everything into memory all at once, by using a Blob and FileReaderSync.

There is example code in the emscripten test suite.

[hsgarciacaballero]

Thank you both for your quick replies!

I have tried WORKERFS and it works perfectly! Indeed, the file is not copied into memory and the C code is able to access and read it. I think it would be interesting to add a simple demo to some page with examples (the one @kripken linked is nice to grasp the basic elements, but rather complex to get the whole picture as it is just test code) so other people can benefit from it.

Is there a repository with examples so I can submit some of the code I have been testing?

[kripken]

@hsgarciacaballero I don't think there's a repo with examples like that, but that's an interesting idea.

Another option might be to add a documentation page to emscripten? (site/ directory in the main emscripten repo has the website docs)

[guest271314]

Is the requirement possible with WebAssembly on a 32-bit system, without using Emscripten?

Note, at Chromium directories and files can be written to local filesystem using Native File System.