Discussion regarding a WASI embedding API (wasi-c-api)

[peterhuene]

Moving the discussion from WebAssembly/wasm-c-api#122 to this repo.

The Wasm C API allows embedders to embed a Wasm engine to instantiate Wasm modules, providing imports to the module.

However, the Wasm C API has no understanding of WASI's existence and that's probably for the best. As @sbc100 and @rossberg suggested, perhaps there is a need for a "WASI C API" for embedding WASI-implementing WebAssembly engines. This would allow embedders to reuse the engine's WASI implementation without having to import their own when using the Wasm C API.

This API would likely be an extension to the Wasm C API that would allow:

• The configuration of WASI (preopens, argv, environ, etc.) on a per-module basis.
• The ability to instantiate (see wasm_instance_new in the Wasm C API) a module with WASI imports (all or some supported subset).

While this doesn't pertain to WASI itself, I think this repo is the best place to kick off a discussion on this topic.

[devsnek]

For Node.js we've made our own WASI impl here: https://github.com/cjihrig/uvwasi

it's fairly runtime independent thanks to using libuv and such.

[peterhuene]

Nice!

However, this isn't about a runtime-independent WASI implementation, but the ability for an embedder of a Wasm engine to request the use of a Wasm engine's WASI implementation if it already comes with one.

What I want to avoid is for the embedding API to force an embedder to have to link in a third party WASI implementation (which can be onerous to do when interacting with the Wasm C API) or, at worst, roll their own WASI implementation.

There should be some standard mechanism of saying "hey Wasm engine, you already provide a WASI implementation, let me use yours!".

[sbc100]

It seems to me that https://github.com/cjihrig/uvwasi would be quite a good candidate for exposing this API no? They already expose a custom API that looks a little like what we might want. @peterhuene did you see the example in the README for uvwasi? Seems like wasmtime and other WASI implementations could implement something like that, no?

[peterhuene]

Agreed that uvwasi's API might be a good basis for this hypothetical WASI C API definition since it definitely exposes configuration points that would be needed to properly configure the engine's WASI implementation.

However, from the embedding point of view, I would want this to sit on top of the Wasm C API to minimize the work needed to instantiate a module with the Wasm engine's WASI implementation imported.

[MarkMcCaskey]

uvwasi looks quite nice! Wasmer recently exposed WASI in its C bindings, but we'd be interested in adding a standard way to do it, too.

I do have a few concerns with the specifics of uvwasi though. For example, limiting the configuration of the WASI filesystem to preopen dirs and such will mean that users of Wasmer will end up using non-standard functions for more interesting use cases of embedded WASI, to the extent that it would almost completely defeat the purpose of standardizing this. One way to fix this that I can think of right now, is to have some kind of opaque handle to the WasiFS that can be operated on.

[sbc100]

@MarkMcCaskey if I understand correctly what you are saying then I agree. The way in which the host exposes filesystems should be flexible and not inherently based on an actual host's filesystem at all.

[MarkMcCaskey]

Yeah, that's a common use case that we're likely to see. My thoughts are that, in general, I'd like to see standardized APIs like this be split into layers:

• the lowest layer is something that all runtimes can easily support
• the next layer is something that almost all runtimes can support
• the next layer is something that most runtimes can support
• ...

This will let us fight fragmentation despite there being a truly diverse set of use cases.

What this really means is designing the lowest common denominator in a way that's generic enough that the other layers can be defined on top of them so that being standard is not all or nothing. For example if a runtime has a feature that is fundamentally incompatible with every other runtime (embedded and constrained use cases are probably where this will show up), then it would be unfortunate if that meant that porting code originally built for the specialized one to a new runtime meant learning entirely new abstractions and rewriting everything -- which was required because the standard APIs were too restrictive to be usable at all by the specialized runtime.

This would in practice turn into a tree of compatibility as the higher layers have more fragmentation.

And the downsides are that:

• it could make the API worse if it's overly generic
• it could lead to over-standardization of things not worth standardizing and lead to a lot of bike shedding

I haven't completely thought this through yet, but designing this such that partial compliance is easy seems important for ensuring this is used widely.