How can I have []byte array in a struct that is passed as my Context

[fproulx-dfuse]

Summary

I need to pass some byte[] array in addition to a few other values as my Context data. The unsafe Pointer thing with CGO seems to limit passing pointers with things other that value types.

Do you know of a workaround to allow passing more complex context object around.

[Hywan]

Unfortunately, cgo has some runtime constraints about pointers passed to C. Everything is explained here https://tip.golang.org/cmd/cgo/#hdr-Passing_pointers.

If you share your exact usecase, maybe I can help?

[attente]

Hi, I think I'm running into a similar issue as the reporter. At a high level, I'm trying to write a wrapper that will use go-ext-wasm to wrap a wasm module generated in rust via wasm-bindgen. Modules produced with wasm-bindgen require some external implementation of a heap, so that needs to be passed around as context data to the import implementations, but can't be done because of the pointer passing problem.

[attente]

To add to the above, wasm-bindgen module imports might also need to call back into the instance itself in order to allocate slices of the module memory, for example. So the context data would also need to include a reference to the module instance itself, so that the implemented imports can call back into the instance.

[Hywan]

Can you give an example so that I could replicate your issue please?

[attente]

I think it's as simple as doing:

instance.SetContextData(unsafe.Pointer(&instance))

which results in:

panic: runtime error: cgo argument has Go pointer to Go pointer

If there isn't a way to include the original instance in the context data somewhere (whether directly or as part of a larger structure), there isn't a way to call its exports from within one of its imports.

[Hywan]

there isn't a way to call its exports from within one of its imports.

Sorry, I don't understand.

[attente]

For example, wasm-bindgen expects an import with the following signature:

func __wbindgen_json_serialize(context unsafe.Pointer, memStrDst, heapObjSrc int32) {
    // ...
}

The implementation of this function is expected to allocate memory for the JSON serialization of a heap object. wasm-bindgen exports a function called __wbindgen_malloc that can be called to allocate that memory. To call this, you would normally do:

instance.Exports["__wbindgen_malloc"](size)

for the current instance. So basically you need access to the current instance from within your implementation of __wbindgen_json_serialize. Ideally this could be passed in as instance context data via instance.SetContextData(unsafe.Pointer(&instance)) so that you could just do:

func __wbindgen_json_serialize(context unsafe.Pointer, memStrDst, heapObjSrc int32) {
    // ...

    instanceContext := wasmer.IntoInstanceContext(context)
    instance := *wasmer.Instance(instanceContext.Data())
    instance.Exports["__wbindgen_malloc"](size)

    // ...
}

But that leads us to the problem above where cgo prevents passing the instance pointer to C.

[losfair]

The safe way of "passing" Go objects to cgo is to have a global array of Go objects and pass the index to cgo.

A fast but unsafe way is to somehow obfuscate the pointer at cgo boundaries, for example by bit-flipping. You need to make sure the pointer is kept somewhere else so that it won't be collected by the GC too early.

[attente]

The safe way of "passing" Go objects to cgo is to have a global array of Go objects and pass the index to cgo.

Thanks, I suspected this was the case and was about to give it a try. I'm still hoping it's avoidable via some other approach but I guess those are the limitations we're dealing with.

[Hywan]

Can you try with the latest version of master please? #85 is likely to address your need.

[Hywan]

Closing the issue. Please re-open if #85 doesn't solve your issue. Thanks!