RFC: Redesign Wasmtime's API

[alexcrichton]

Rendered

---

Overhaul the wasmtime crate's API to improve it along a number of vectors:

• Greatly improve the multithreading story in all languages (Rust, C, Go, ...),
  namely enabling safe usage of wasmtime in multithreaded server runtimes
  where wasm objects can migrate between threads.
• Leverage mutability in Rust to allow &mut access to user-defined state
  instead of requiring users to use interior mutability.
• Simplify memory management in the C API and embeddings.

The only major cost relative to today's API is that ExternRef will move to
atomic reference counting and will require T: Send + Sync on constructors.

[bnjbvr]

A few preliminary comments, but this looks great!

[bnjbvr]

Above it is suggested that having the Store be Send + Sync is optional and depends on T being Send + Sync too; so is this a strong requirement, even when the embedder doesn't need a Send + Sync Store?

[alexcrichton]

That's correct, regardless of whether the embedder needs Send and Sync host functions will always be required to be Send and Sync. This is thought to be ok, though, because most host state should be stored in T rather than in the funtion itself, and it's expected that the majority of host functions are simply 0-sized closures that are just code and no data.

[bnjbvr]

Are these two functions doing more than what the AsContext/Mut trait methods provide?

[alexcrichton]

Nah they're just explicitly named so if users want to call them they don't have to import the trait. I'm not sure if this makes sense but it's always an option we can have of course!

[acfoltzer]

I'm so excited about this, thank you for making it happen!

I'm a little unclear about the role of the store's type variable T from a library user perspective. I think it would help to have some examples of user code showing how the T impacts the developer experience. Is it usually going to be inferred or fixed by library code, or will the user have to reason about what this variable should be?

[alexcrichton]

A good question! It's sort of tangentially touched on here but the general idea is that if you are a library crate then to work with a Store<T> you'll probably place some sort of trait bound on T to get access to the data you'd want. For example with WASI:

// in the wasmtime-wasi crate
fn add_to_linker<T>(funcs: &mut Linker<'_, T>)
    where T: AsMut<WasiCtx>;

or alternatively:

// in your crate
fn add_to_linker<T>(
    funcs: &mut Linker<'_, T>,
    get: impl Fn(&mut T) -> &mut MyLibraryState + Copy + Send + Sync + 'static,
);

The idea being that the owner of the store will still determine the T for you. Libraries will work generically with some T with trait bounds or accessors or similar.

In terms of application code and such I'd imagine that the T is generally inferred from a call to Store::new for you, given that you're selecting some type to be owned by the store there.

[acfoltzer]

Hmm, I think I get it. To check my understanding, if I want to have my own application-specific context type but also a WASI context, how might that look? Create my own combined context type that impls both AsMut<WasiCtx> and AsMut<MyAppCtx>?

[alexcrichton]

@acfoltzer indeed! While it's not the prettiest this is an example in the branch I'm working on. The wasmtime CLI optionally supports some WASI proposals both at runtime and compile time, and that struct is used to manage the state where it's dynamically initialized and then used by linker-added functions through the BorrowMut traits

[acfoltzer]

Perfect, that makes tons of sense. Thank you!

[bjorn3]

Suggested change

	it's simply an index into the `Store`. This implementaiton detail is how the
	it's simply an index into the `Store`. This implementation detail is how the

[alexcrichton]

I've updated with a tweak to the APIs in languages like Python, Go, and .NET. Originally I intended that "pass the store around" wouldn't be as common there, but the behavior of the GC in Go and converting a wasmtime_val_t to a wasmtime_func_t has tipped my opinion in favor of "pass the store everyhwere" like what happens in C & Rust.

[alexcrichton]

Motion to finalize with a disposition to merge

We've reached out to various Wasmtime embedders, and have received very positive feedback on the changes proposed here. Additionally, we have full implementations of the proposal for Wasmtime's Rust and C-APIs, wasmtime-py, and wasmtime-go.

We also analyzed the impact on various existing embeddings, and found it to be very reasonable in all cases.

As such, we proposing finalizing this RFC with a disposition to merge it.

Stakeholders sign-off

Given the broad impact of these changes, we decided to also tag a broad list of contributors and Wasmtime embedders for feedback.

Arm

• @akirilov-arm

DFINITY

• @granstrom

Embark Studios

• @bnjbvr
• @repi

Fastly

• @cfallin
• @sunfishcode
• @jedisct1
• @fitzgen
• @pchickey
• @peterhuene
• @acfoltzer
• @iximeow

Intel

• @mingqiusun
• @abrown
• @jlb6740

Microsoft

• @thomastaylor312
• @bacongobbler
• @radu-matei

Mozilla

• @julian-seward1
• @yurydelendik

IBM

• @uweigand

wasmCloud

• @autodidaddict

Unaffiliated

• @bjorn3

[acfoltzer]

• @acfoltzer

• @acfoltzer

[alexcrichton]

Entering Final Call Period

https://github.com/bytecodealliance/rfcs/blob/main/accepted/rfc-process.md#making-a-decision-merge-or-close

Once any stakeholder from a different group has signed off, the RFC will move into a 10 calendar day final comment period (FCP), long enough to ensure that other stakeholders have at least a full business week to respond.

The FCP will end on Thursday June 3.

[sunfishcode]

Could you briefly discuss how this design could be extended to support a hypothetical native wasm threads proposal? Is this just a matter of using a T that has Send + Sync, or would this need additional support?

[alexcrichton]

I think it may be a little too early to say how we might support a proposal like that. The discussion on that issue is largely about thread id representation which is somewhat orthogonal to the embedding API. For the embedding API though the proposal is so vague at this point I'm not sure what our design constraints are. For example with a hypothetical thread.new instruction:

• I'm not sure how the imports of the current module are handled. For example if you imported a function, how is that fuction import satisfied on the other thread? Similarly if you have a function table in the module it's not clear how that would be "forked" into another thread. This might mean recursively instantiating a whole bunch of modules, but you'd also bottom out in host closures and be faced with a question of how to "fork" them to new threads too.
• I'm not sure how globals would be handled. Current globals, like the stack pointer, naturally don't want to be shared amongst threads and the sub-thread would want to get a fresh copy. (but where does the initial value come from?)

I'm sure there's other questions too, but without knowing much about the proposal it's hard to say what the embedding API might look like.

My rough mental model for the current wasm threads proposal (not to be confused with that issue for a native wasm thread) is that there'd be a Store-per-thread. Here T wouldn't need Send or Sync since the embedder would simply arrange the Store to be on its own thread somewhere else. Trying to fuse this with a native wasm threads proposal sort of looks like we'd need like a "store factory" where there's a built-in ability to create a Store on another thread. That namely does indeed need to create a T, and that could presumably be done with a custom trait/helper or even just T: Clone. This would indeed require T: Send. I don't think that T: Sync would make sense since we provide mutable access to the T which means that we can't share the T across threads anyway.

Overall I don't think we're necessarily in a bad position to support a proposal like that, but it's tough to say given the current state. I do think that we're still better off than we are today where host state is littered across all sorts of host functions and such. The only state in a Store which is maybe not Send and Sync is the T itself. This means that all reasoning about thread-safety is entirely tied to T since everything else is threadsafe (e.g. host closures are all Fn() + Send + Sync).

Currently there's a bunch of pieces floating around where if you connect a few and squint hard it sort of looks like an implementation of a native threads proposal, but I think there needs to be more concrete design planning and such done first.

[thomastaylor312]

I think this API will be very nice for consumers. The trade offs and concerns are well documented as well. Really great work here!

[bnjbvr]

Great overhaul, it makes manipulating the API much more explicit and clear to reason with in multithreaded contexts!

[repi]

Good stuff! 👍

[alexcrichton]

Ok the FCP period has now finished and no objections have been raised. Time to merge then!