NODEFS not compatible with Deno when used through Pyodide

[JanPokorny]

This is an issue I have originally rised with Pyodide here: pyodide/pyodide#4647

As Pyodide utilizes NODEFS directly from Emscripten, the maintainers suggested I raise this bug here.

---

This is an example Node.js program that uses Pyodide to write a file in a NODEFS-mounted folder:

// test.mjs
// run with: node test.mjs

import { loadPyodide } from "pyodide"

const pyodide = await loadPyodide()

pyodide.setStdout()
pyodide.setStderr()

pyodide.FS.mount(pyodide.FS.filesystems.NODEFS, { root: "." }, "/home/pyodide")

await pyodide.runPythonAsync("from pathlib import Path; Path('out.txt').write_text('Hello, World!')");

This works as expected, the file out.txt with contents Hello, World! is written to current folder.

---

The same code, adjusted to run with Deno:

// test.ts
// run with: deno run --allow-write --allow-read --allow-net test.ts

import { loadPyodide } from "npm:pyodide/pyodide.mjs"

const pyodide = await loadPyodide()

pyodide.setStdout()
pyodide.setStderr()

pyodide.FS.mount(pyodide.FS.filesystems.NODEFS, { root: "." }, "/home/pyodide")

await pyodide.runPythonAsync("from pathlib import Path; Path('out.txt').write_text('Hello, World!')");

This, however, crashes with the following output:

error: Uncaught (in promise) PythonError: OSError: raw write() returned invalid length 2831100 (should have been between 0 and 13)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/lib/python311.zip/_pyodide/_base.py", line 573, in eval_code_async
    await CodeRunner(
  File "/lib/python311.zip/_pyodide/_base.py", line 393, in run_async
    coroutine = eval(self.code, globals, locals)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<exec>", line 1, in <module>
  File "/lib/python311.zip/pathlib.py", line 1078, in write_text
    with self.open(mode='w', encoding=encoding, errors=errors, newline=newline) as f:
OSError: raw write() returned invalid length 9188352 (should have been between 0 and 13)

    at new_error (file:///Users/jp/Playground/pyodide-test/node_modules/.deno/pyodide@0.25.0/node_modules/pyodide/pyodide.asm.js:9:10014)
    at <anonymous> (wasm://wasm/02257296:1:1412707)
    at <anonymous> (wasm://wasm/02257296:1:1441695)
    at _PyEM_TrampolineCall_JS (file:///Users/jp/Playground/pyodide-test/node_modules/.deno/pyodide@0.25.0/node_modules/pyodide/pyodide.asm.js:9:120166)
    at <anonymous> (wasm://wasm/02257296:1:1717397)
    at <anonymous> (wasm://wasm/02257296:1:2663102)
    at <anonymous> (wasm://wasm/02257296:1:1983263)
    at <anonymous> (wasm://wasm/02257296:1:1718654)
    at <anonymous> (wasm://wasm/02257296:1:1719359)
    at <anonymous> (wasm://wasm/02257296:1:1719521)
    at <anonymous> (wasm://wasm/02257296:1:2525605)
    at <anonymous> (wasm://wasm/02257296:1:2549761)
    at <anonymous> (wasm://wasm/02257296:1:1719817)
    at <anonymous> (wasm://wasm/02257296:1:1718937)
    at <anonymous> (wasm://wasm/02257296:1:1439596)
    at Object.Module.callPyObjectKwargs (file:///Users/jp/Playground/pyodide-test/node_modules/.deno/pyodide@0.25.0/node_modules/pyodide/pyodide.asm.js:9:64297)
    at Object.Module.callPyObject (file:///Users/jp/Playground/pyodide-test/node_modules/.deno/pyodide@0.25.0/node_modules/pyodide/pyodide.asm.js:9:65365)
    at Timeout.wrapper [as _onTimeout] (file:///Users/jp/Playground/pyodide-test/node_modules/.deno/pyodide@0.25.0/node_modules/pyodide/pyodide.asm.js:9:32855)
    at cb (ext:deno_node/internal/timers.mjs:64:31)
    at eventLoopTick (ext:core/01_core.js:204:13)

---

Not sure if NODEFS aims to officially support Deno, or if this is even fixable on Emscripten side (maybe it's purely a Deno bug) -- but raising it anyway for awareness.

[sbc100]

Emscripten does not have any official support for Deno, and we don't currently do any testing under Deno.

Does deno claim to be API compatible with node for things like the fs module? If so I don't see why it wouldn't work there. All we do in emscripten is use the normal node fs API as far as I know.

[sbc100]

If you (or anyone else) would like to contribute some level support and testing with deno that would be most welcome.

[JanPokorny]

@sbc100 Deno claims fs compatibility with the following caveats (https://docs.deno.com/runtime/manual/node/compatibility):

Does this look like anything that would affect NODEFS?

In any case I have raised this with Deno as well: denoland/deno#23181

[sbc100]

I don't think our NODEFS code depends on any of those things... but I could be wrong.

It might be good to create a simple/small example of a C program that reproduces this issue.. then perhaps do some tracing debugging to see which NODEFS/fs calls are used.