-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Extend CustomCodeMemory interface allowing for using previously-mapped images #10865
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…d images `Module::deserialize_raw` and `CustomCodeMemory` were introduced with aim of supporting `no_std` platforms. With small tweaks they can also be used for using Wasmtime on platforms that have full-blown virtual memory capabilities, but doesn't allow for directly mapping executable pages from user code instead limiting that capability for system loader. Adding features necessary for such platforms was previously attempted by bytecodealliance#8245. There are currently two issues with using `Module::deserialize_raw` for using images either statically linked into embedder executable or dynamically loaded shared objects: - `CodeMemory::publish` will initially make entire image read-only, destroying executable permissions that cannot be restored by user code. This will happen even if `CustomCodeMemory` is provided. - `CodeMemory::publish` will attempt to unconditionally register unwind information. As these are already properly registered by the system loader this is superfluous at best, or could fail module loading if system decides to return error on attempt for double-registration. This commit solves these issues by: - Moving responsibility for making image RO to `CustomCodeMemory` hook. - Making `CustomCodeMemory` publishing hook return enum that tells what steps (only mapping, or mapping with registration) were performed, using default implementation only for actions not reported by the hook. - Additionally `CustomCodeMemory` hooks also receive pointers to the entire image, not only executable section. This allows the embedder to keep track of the images and unload them in case they were loaded as dynamic shared object.
00d5875
to
180ddad
Compare
Okay, I got a little ahead of myself.
So I still think this makes sense. |
This seems reasonable to me to support, but what do you think of modeling this a bit differently? This is a pretty sensitive piece of the codebase which we want to ensure doesn't accidentally regress and having a lot of intertwined "if this else that" style logic can be relatively confusing to follow. Do you think it would be possible to internally model this as "assume there's always a custom memory publisher"? That way adding more hooks would mean adding more trait methods, and the "driver" would still be relatively straightforward of "do this, then that, then that". A custom trait could then override any one particular step to a noop and that wouldn't involve a need for custom enums/etc which are pretty tightly bound to the exact structure of what we have today as well. |
Do you think about trait methods that would return bool indicating whether default implementation should be called, or calling only trait methods with default implementation provided as default CustomCodeMemory? In second case should MmapVec become part of public API? |
Ideally we'd model everything with the trait to allow embedders to customize all the hooks necessary, but that's also not easy as there's internal types, like |
Module::deserialize_raw
andCustomCodeMemory
were introduced with aim of supportingno_std
platforms. With small tweaks they can also be used for using Wasmtime on platforms that have full-blown virtual memory capabilities, but doesn't allow for directly mapping executable pages from user code instead limiting that capability for system loader. Adding features necessary for such platforms was previously attempted by #8245.There are currently two issues with using
Module::deserialize_raw
for using images either statically linked into embedder executable or dynamically loaded shared objects:CodeMemory::publish
will initially make entire image read-only, destroying executable permissions that cannot be restored by user code. This will happen even ifCustomCodeMemory
is provided.CodeMemory::publish
will attempt to unconditionally register unwind information. As these are already properly registered by the system loader this is superfluous at best, or could fail module loading if system decides to return error on attempt for double-registration.This commit solves these issues by:
CustomCodeMemory
hook.CustomCodeMemory
publishing hook return enum that tells what steps (only mapping, or mapping with registration) were performed, using default implementation only for actions not reported by the hook.CustomCodeMemory
hooks also receive pointers to the entire image, not only executable section. This allows the embedder to keep track of the images and unload them in case they were loaded as dynamic shared object.