🛤 Unmanaged closures

[binji]

This is a tracking issue for a post-MVP feature
It will be updated as the issue progresses.

Topic Unmanaged Closures
Champion Mark Miller @erights
Status withdrawn
Phase pre-proposal
Linked issues
Linked repositories

• https://github.com/WebAssembly/unmanaged-closures
• https://github.com/erights/wasm-linkage/tree/master/proposals/wasm-linkage

Details

Many applications are extensible frameworks, built to accept third party plugins that add value. For example, Photoshop accepts third party plugins for image transformations. Google EarthEngine is a web-based application that accepts third party plugins for geospatial computation. Many of these plugins are fallible; EarthEngine wishes to protect its own integrity from the potential corruption or compromise of its plugins. At the same time, EarthEngine needs to expose a rich API to these plugins so that they can interact flexibly with EarthEngine. Currently, EarthEngine is written in JavaScript and enforces that its plugins are written in SES (Secure EcmaScript) an ocap subset of JavaScript.

Wasm would enable EarthEngine to be written in C++, and to accept plugins written in C++. Wasm already provides all the protection mechanisms needed so they can be linked together in one OS address space and interact via full speed function calls. EarthEngine would reside in one compartment (a set of wasm modules linked together sharing the same memories and tables --- see below) and each plugin in another. EarthEngine would export the functions it wishes to make available to its plugins, and instantiate each plugin only with access to these exports. The code within each compartment is as vulnerable to buffer overflow or other memory corruption as we expect of C++. But this separation prevents a memory corruption vulnerability in a plugin from compromising EarthEngine.

In this sense, each wasm compartment resembles an OS process with its separate address space and descriptors; and an inter-compartment call resembles an IPC. However, the cost of the context switch between these protection domains is simply the cost of code in one wasm compartment calling an imported function that was exported by another linked wasm compartment.

This demonstrates that wasm already provides the safety needed for such protection. But it lacks the needed expressiveness.

[chicoxyzzy]

Should we remove this proposal from proposals list? It seems that it was superseded by typed function references proposal

[erights]

Not quite superseded. The reference types proposal does not enable a wasm instance to pass a dynamic number of encapsulated functions, i.e., there's no way to emulate closures safely. Now that WASI is proceeding, and will be defining capability-oriented APIs, this would fill a need that isn't filled by reference types by themselves.

However, this has been on hold because it would be fully superseded by wasm-gc. If the gap between reference types and full wasm-gc is long, this proposal would fill a need. However, the closures should have good compatibility with the managed closures that will come with wasm-gc, which remains to be settled.

[rossberg]

@erights, the recent proposal sketch for typed function reference may generalise function references to full (opaque) closures, via a possible func.bind instruction. Would that address your use case?

[erights]

@rossberg
After @lars-t-hansen explained them to me (thanks!) I had another look at func.bind at https://github.com/WebAssembly/function-references/blob/master/proposals/function-references/Overview.md . Not a lot of detail there, but I believe the answer is essentially yes.

My concern is that the generality of these closures implies heap allocation and gc. AFAIK, wasm itself including reference types, ignoring binding to a JS host, never requires implicit dynamic allocation and gc. The proposal here is essentially the special case of func.bind where only a single integer parameter is bound. The reason for calling out this special case was the hope that these "unmanaged closures" could be passed by value, by passing a wider value. However, @lars-t-hansen explained to me why this is unlikely to happen.

In any case, yes, func.bind is close enough that this proposal is not pointless. I withdraw it. How does one shut down this proposal officially?

[rossberg]

@erights:

AFAIK, wasm itself including reference types, ignoring binding to a JS host, never requires implicit dynamic allocation and gc.

Well, yes and no. It depends on what exactly you define as "Wasm itself". The intro to the proposal mentions:

In a Wasm engine, function references (whether first-class or as table entries) are already a form of closure since they must close over a specific module instance (its globals, tables, memory, etc) while their code is shared across multiple instances of the same module. It is hence expected that the ability to form language-level closures is not an observable extra cost.

While module instantiation (currently) is only accessible through the API, it is part of Wasm itself. However, one could imagine a "static" engine that does not support instantiating modules dynamically and thus keeps all entities alive forever.

It is also worth noting that there are other proposals that make some amount of automatic memory management in engines unavoidable, most notably the exception proposal, which must manage the life times of a caught exception and associated meta data.

In all these cases, we assume that reference counting (a.k.a. poor man's GC), would be good enough as an implementation strategy.

[chicoxyzzy]

How does one shut down this proposal officially?

We can just close this issue, delete repository and remove proposal from proposal table. Should removal be discussed at the next meeting to be sure everyone is ok with it?

[erights]

In all these cases, we assume that reference counting (a.k.a. poor man's GC), would be good enough as an implementation strategy.

I cannot be enough for general closures, because you could have a cycle of such closures that refcounts cannot collect. But this is not an objection. I am happy with this direction.

We can just close this issue, delete repository and remove proposal from proposal table. Should removal be discussed at the next meeting to be sure everyone is ok with it?

Sure. If it does not violate our process, I'm perfectly happy for us to just delete it now.

[rossberg]

@erights, I don't think the proposal provides a way to construct cycles among closures, because the closure environment isn't mutable.

[erights]

@erights, I don't think the proposal provides a way to construct cycles among closures, because the closure environment isn't mutable.

I see! In this case, a language-level closure (for a language with mutable closures) would still be an unmanaged closure in the relevant sense. A cycle of language level closures would be hooked up indirectly by indexes into function tables, where wasm sees those only as closures binding numbers. For those cycles, it would be up to the relevant language runtime to deallocate these cycles, or not, as part of that language's own storage management strategy. As far as wasm is concerned, the function tables are roots, and the closures in them are legitimately not considered garbage.

Cool! I like it. Thanks.

[erights]

After @lars-t-hansen explained them to me (thanks!)

OMG, sorry. Weird finger error. I meant @lukewagner . Thanks Luke!

[binji]

We can just close this issue, delete repository and remove proposal from proposal table. Should removal be discussed at the next meeting to be sure everyone is ok with it?

I think we can close the issue, but I'd rather not delete the repository, since we'd lose history if we do. We can mention it at the next meeting, but I don't think it's necessary to ask the group about it.

[erights]

Thanks.

[chicoxyzzy]

@binji repo is empty. It doesn't have any files and open / closed issues.

update: I double checked, we link to this issue and not repo everywhere so I believe that it's safe to delete repo