Weak imports #36
Comments
sunfishcode
added
discussion
|
I would assume that calling |
|
One solution is to support nullable references only for importing variables/functions as a subset of the GC proposal. That seems trivial enough that I would expect all WASM implementations to be able to easily implement that, since it doesn't actually require GC. |
|
this definitely seems more like a general wasm problem than a wasi problem. i do like what programmerjake said. |
I'm definitely raising my hand for not making WASI dependent on a garbage collector. GC is meant to be fully optional and an efficient implementation will be a significant code base compared to the rest of WASM runtime. In that sense, I'm supporting what @programmerjake proposed as it's exactly as @devsnek says - not a WASI problem, but rather a WASM problem. Btw. I find any naming convention as wrong solution when proposed from the beginning as to me naming conventions try to solve rather some problems discovered later - and having two such conventions already in the standard makes a really bad impression. In my opinion weak imports shall be first class citizens in WASI/WASM. |
|
FWIW, there has been a general sentiment that the typed function references proposal, which brings typed references into Wasm, should also include nullable ones. While I didn't have time yet to update the proposal, that's the current plan. So nullable refs are to be split off from the GC proposal already. Whether they are the best way to provide feature detection is a different question. Dynamic feature detection is a dead end anyway, since you cannot opt into types that way. That, however, would be necessary for APIs that introduce new host types (e.g., imported reference types). So in general, you'll need link-time feature detection. |
|
I can think of two other possible solutions:
These could then be compared with 0 (or some other fixes, invalid, table entry). |
|
The general case I have in mind is an API that defines its own host reference types. Consider a file system module for example, which might provide a (Also, I'd argue that the implication of requiring call_indirect for every API call is defeating half the purpose of the Wasm module system.) The problem with dynamic feature checking in general is that it conflates program execution with program configuration. For scalable modular software it's better to keep those separate -- at least that's what all larger software systems do. Feature testing is a hack that was invented in languages that did not have any proper support for modularity or versioning at the time and where programs remained small enough to keep that tractable. |
|
If I understand Dan's proposal correctly, the ".weak" and ".is_resolved" suffixes would require no changes to wasm's import mechanism, it is a just a convention in WASI. (Presumably, as @programmerjake suggested, trying to call a non-resolved weak import would trap). That seems relatively clean, and preferable to crossing into the dynamic world of nullable function references. Those have some overhead, both in metadata and possibly runtime, whereas imports have less overhead since they are only bound once at instantiation. And I agree with @rossberg that import calls via references and indirection seem like the wrong workaround. |
|
Surely calling It does avoid the call_indirect and allow direct imports I agree is preferable. On the downside it does double the number of imports and require the embedder to create fake trapping functions for each weakly undefined functions (one per signature I guess?) |
|
Yes, I meant to say above that an unresolved ".weak" function would trap. And yes, ".is_present" enables dynamic feature detection. This is useful. Requiring programs to be split into modules for each API feature they need to detect would be a considerable burden, even on programs written in languages with modern module systems. I even wonder whether we could extend the ".weak" runtime detection to work for types too. Rough idea: Allow ".weak" on type import names, and then have any imported function with a signature that mentions an unresolved weak-import type trap. Unresolved weak-import types would otherwise act like |
|
I don't really understand the interest of a specialized weak import mechanism, given the possibility of a more general optional type for imports. Would .weak and .is_present have use cases that optref wouldn't cover? |
|
My understanding is that this proposal is designed to work in lieu of a first class weak import in the core spec and that we could/should move over to spec'd weak imports once such a thing exists. Its also a goal to avoid the extra level of indirection involved with |
|
The weak import mechanism could be slightly faster since you can codegen a stub that traps instead of performing a null check, and you can use a direct jump. |
|
@binji If the |
|
You wouldn't be able to do static analysis on a |
Isn't that something VMs could also do with function optrefs? I don't think there's anything in the proposal that says that null refs have to be encoded as a 0x0 pointer.
Not sure how relevant this is then. Given their respective age and complexity, I'd expect the funcref proposal to be standardized way before WASI. |
|
Indeed, as @binji mentions, the fastest option would be the weak import mechanism where the embedder provides a stub that traps. The call sequence would be exactly the same as other imports; a load from an internal array and an indirect call. |
|
Another concern with the optref in this context is that it isn't composable. It isn't "Option T" for any T, so it doesn't easily support things like optional i32 or optional optional funcref. And as @rossberg pointed out, it doesn't support other kinds of entities like imported types. While "WASI core" is entirely functions today, on one hand WASI will grow and include other kinds of things in the future, and on the other, API feature detection will be needed in many more places than just WASI. Weak imports still leave the question of what to do with things other than functions. Perhaps:
This may be pointing in the direction of adding weak imports to the wasm core spec. |
I wrote up this as initially proposed by Dan, using name mangling rather than a custom section. I believe there is still room to switch a custom section if people feel strongly about it. The main downsides I see of using such mangling are: 1. It is that reserves the ".weak" and ".is_present" suffixes 2. It might not play nicely with possible future attributes we might want to add to imports. Fixes #36
I wrote up this as initially proposed by Dan, using name mangling rather than a custom section. I believe there is still room to switch a custom section if people feel strongly about it. The main downsides I see of using such mangling are: 1. It is that reserves the ".weak" and ".is_present" suffixes 2. It might not play nicely with possible future attributes we might want to add to imports. Fixes #36
I wrote up this as initially proposed by Dan, using name mangling rather than a custom section. I believe there is still room to switch a custom section if people feel strongly about it. The main downsides I see of using such mangling are: 1. It is that reserves the ".weak" and ".is_present" suffixes 2. It might not play nicely with possible future attributes we might want to add to imports. Fixes #36
|
I've been thinking about and alternative to #47 based a custom section rather than magic name mangling. My initial plan was to enumerate all the weak import in a list. Each element would be module and and function import name, and a corresponding global import to signal presence of he function. One of the downsides of this approach is that it requires 3x as much space in the binary for each weak import (since it names once in the import section and twice in the custom section). It also doesn't avoid the name mangling issue, since the "is_present" global will still need to be added to the imports section, possibly colliding with existing imports. So using a custom section doesn't get us away from adding some form of name mangling. Also, given a name mangling scheme such as the ".is_present" and/or ".weak" suffixes its already simple for the embedder to enumerate all the weak imports, and therefore enumerating them in the custom section seems redundant. So here are the current directions that I think could work:
Right now I'm leaning back towards (2) or possibly (3) (in order to avoid spec'ing the name mangling). Thoughts? |
|
@sbc100: I'm curious why the import functions and globals need to be specified by name in the custom section, rather than by index |
|
is it unlikely that WebAssembly/design#1281 is going to move forward? |
Good question, I seem to remember @lukewagner had opinions on this. It seems like it would be strange to invent an import index space, but I suppose we could use function and/global index space. Luke can you remember your arguments against using indexes here? |
Not in the timeframe that we want for WASI I fear. Also, that fact that we can do this in userspace doesn't help our case for building something first class. |
|
As long as it is planned to move forward, I feel like we should work on that instead. Coming up with our own system seems antithetical to Wasm's design goals. |
Index spaces are an internal naming mechanism, and as such, an implementation detail of a module that is not observable externally. A custom section mechanism using them would require special access to module internals and would not be polyfillable. |
|
Got it, thanks @rossberg. I am happiest with option 4 then - I'd prefer to put names in the custom section than defining some way to mangle names. |
I'm not sure I follow this logic - what is the primary friction seen to be? For example implementing this in Node.js would be trivial. |
I wrote up this as initially proposed by Dan, using name mangling rather than a custom section. I believe there is still room to switch a custom section if people feel strongly about it. The main downsides I see of using such mangling are: 1. It is that reserves the ".weak" and ".is_present" suffixes 2. It might not play nicely with possible future attributes we might want to add to imports. Fixes #36
I wrote up this as initially proposed by Dan, using name mangling rather than a custom section. I believe there is still room to switch a custom section if people feel strongly about it. The main downsides I see of using such mangling are: 1. It is that reserves the ".weak" and ".is_present" suffixes 2. It might not play nicely with possible future attributes we might want to add to imports. Fixes #36
Wasm doesn't have a "weak" import mechanism, which would allow one to
(import "foo" "bar")where "bar" may or may not exist in "foo" (or perhaps where "foo" itself may not exist). This would be one way to allow feature detection in APIs.As one possible solution, it's expected that the wasm GC proposal will eventually give wasm the concept of optional/nullable references, which would permit a kind of weak import where one imports global variables with reference type which can be either references to the desired entities or absent/null. However, GC may not be available in wasm for some time, and even when it is, it may not be implemented in all wasm engines, so it's not entirely obvious whether we should depend on it for something so fundamental to WASI.
Another is to essentially add a weak-import system on top of Wasm. For example, we could use a naming convention, such as a ".weak" suffix, to indicate a weak import, and a ".is_present" suffix on a global variable import to hold a boolean indicating whether a weak import was successfully resolved, like this:
A similar scheme could enable detection of modules too:
or even both at the same time:
Another option would be to have a dynamic API that takes a string, however while we can do that, passing around strings introduces some complexity and it seems like if have a naming convention for ".weak", we might as well have ".is_present" too.
I'm not attached to any of the details here, I'm mainly just looking to start a discussion.
The text was updated successfully, but these errors were encountered: