Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native plugins support #154

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft

Native plugins support #154

wants to merge 10 commits into from

Conversation

tqwewe
Copy link
Member

@tqwewe tqwewe commented Nov 15, 2022

Adds support for native plugins (.dll/.so/.dylib) which can be loaded with the --plugins flag.

An example exists in the examples/ directory, along with some docs on running the example.

Related to #69 (nice)

@@ -131,6 +138,8 @@ impl ProcessState for DefaultProcessState {
wasi_stderr: None,
initialized: false,
registry: self.registry.clone(),
plugins: self.plugins.clone(),
plugin_state: HashMap::new(),
Copy link
Member Author

@tqwewe tqwewe Nov 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we clone the plugin state here? Or put the whole plugin state in an Arc?
The only problem is that we cannot downcast_mut when the state is in an Arc, or even borrow it mutably, and I'm afraid we might need to use mutex's in this case.

@@ -456,11 +477,31 @@ impl DistributedCtx<LunaticEnvironment> for DefaultProcessState {
wasi_stderr: None,
initialized: false,
registry: Default::default(), // TODO move registry into env?
plugins: Arc::new(vec![]),
plugin_state: HashMap::new(),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should plugins and plugin state be empty in new_dist_state?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not (I'm not sure 😄 ), when reading through #69 the original idea was to have different plugins for each Environment.

Testing plugins or comparing performance between competing implementations becomes trivial, just spawn different environments, attach different plugins to them and run the same code.

When spawning via distributed API we use Environment to crate state.

If I understand this correctly calling a function from a dynamic library will fail, in this case when spawning a process on a different node, since plugin is not linked?

@withtypes withtypes mentioned this pull request Nov 16, 2022
@tqwewe tqwewe marked this pull request as draft November 17, 2022 03:50
@tqwewe
Copy link
Member Author

tqwewe commented Nov 17, 2022

I just came across an issue, and I'm not sure whats going on.

Screen Shot 2022-11-17 at 12 05 15 pm

When I run lunatic in release, everything works fine... but running lunatic in debug mode causes the host functions to silently fail 🤔

It seems like if I compile the wasm plugin and module to debug mode, and run lunatic in debug mode, it works then.
So I cannot run lunatic with a wasm module that doesn't match the release/debug target.

@bkolobara
Copy link
Contributor

Sorry, it took me a bit longer to get around and check this out. Great work so far!

I also can't figure out what the issue with release/debug builds could be. From my understanding on how dynamic loading works, it shouldn't be an issue. It might be something Wasmtime specific, because they do some assembly magic to allow stack switching? Maybe creating a minimal reproducible example with just wasmtime and dynamic loading could help us to pinpoint it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants