chicken and the egg linker.func design

[softprops]

I'm running into a problem with the current Linker.func api as it relates to the wiring of a wasm runtime.

The following illustrates a problem I'm having where in order to instantiate a Module, one must configure a linker's functions. In my case this runtime is expected to provide a an implementation of a number of module functions. When they are not defined wasi's call to instantiate a Module fails with very useful error reporting describing the missing implementation and its type signature. However in order to define those linker functions I would require the instantiated modules Memory reference if I understand this system currently.

I'm new to wasm so I may be missing something here but presumably the example below is a more helpful and fill the gaps I'm not sure how to describe yet.

use wasmtime::*;
use wasmtime_wasi::{Wasi, WasiCtxBuilder};
use byteorder::ByteOrder;

let engine = Engine::default();
let store = Store::new(&engine);
let mut linker = Linker::new(&store);
let wasi = Wasi::new(
      &store,
     WasiCtxBuilder::new()
           .inherit_stdout()
           .inherit_stderr()
           .build()?
);
wasi.add_to_linker(&mut linker)?;

// the egg exists here. The egg needs the a reference chicken's memory
linker.func(
       "mod_name",
       "fn_name",
        |offset_arg0: i32| {
           // performs some computation then updates the instances Memory
            let some_computed_result: u32 = ...;
            unsafe {
                    byteorder::LittleEndian::write_u32(
                        &mut memory.data_unchecked_mut()[offset_arg0 as usize..],
                        some_computed_result,
                    );
           }
           0
        },
 )?;
      
// the chicken is born here the chicken can not be born without the linker being defining the implementation
// of `mod_name::fn_name`
let instance = linker.instantiate(&Module::from_file(&engine,"path/to/app.wasm")?)?;

let memory = instance
   .get_memory("memory")
   .expect("all instances should have memory. this one does not");

I'm basing this Rust application off of an implementation in go-wasmtime which orchestrates this all in the the setup of of a structs NewFoo func where the reference to memory can be nil at the time of link definition. This is made slightly more challenging with rusts memory module and references to "self" and/or any external closure references inside linker.func()

I can fill in more concrete details if needed but I'm wondering if it's clear I'm missing something very fundamental right out of the gate with wasmtime's expected api usage.

[Rochet2]

Only one memory is allowed. Both modules would point to same memory if one creates the memory and the other imports it for example.

From what I understand, a host function can have caller: Caller as parameter. This is shown in the examples. The Caller allows you to get an export, for example if the module exports memory you can access it.

I guess one issue I see here might be that you would always need to export the memory in the module if you want the host to be able to access it with this method.

[alexcrichton]

Yes currently this is a limitation of WebAssembly. All instances must be instantiated in a DAG to ensure there's no cycle for instantiation. As @Rochet2 mentions, however, the Caller type in Wasmtime is intended to help solve this issue about memories specifically (there's a longer example on Func too).

Overall there's a few ways you can solve this:

• The module could import memory instead so you, as the host, can create memory early.
• Your closures can use Caller to access the caller's memory
• You can use a combination of Rc and RefCell in Rust to communicate the memory back to the imports after instantiation (it means imports won't work during instantiation, however)
• Eventually the interface types proposal is intended to make this much easier where communication of types in memories will automatically be handled by the host.

[softprops]

Thanks for the fast feedback folks. These were very helpful insights.

I don't have control over changing the wasm apps contact api with it's host environment but a few of the options above sound like tenable experiments.

The caller example looks to be precisely what I need to get a handle on the instances memory.

let log_str = Func::wrap(&store, |caller: Caller<'_>, ptr: i32, len: i32| {
    let mem = match caller.get_export("memory") {
        Some(Extern::Memory(mem)) => mem,
        _ => return Err(Trap::new("failed to find host memory")),
    };

It can't be said enough but the quality of docs in these projects are superb

[softprops]

Closing this for now since I have what I need to move forward.

Thanks for an amazing project!