[20.0.0] More backports of fixes, mostly c-api related (#8368)

* c-api: Better differentiate between `wasm.h` and `wasmtime.h` APIs (#8344)

This renames some types and adds some type aliases to help us better distinguish
between `wasm.h` APIs and `wasmtime.h` APIs, primarily for `Store`-related
types. In general, `WasmFoo` is related to `wasm.h` and `WasmtimeFoo` is related
to `wasmtime.h`.

* `StoreRef` -> `WasmStoreRef`
* Introduce the `WasmStore[Data]` and `WasmStoreContext[Mut]` aliases
* `StoreData` -> `WasmtimeStoreData`
* `CStoreContext[Mut]` -> `WasmtimeStoreContext[Mut]`
* Introduce the `Wasmtime{Store,Caller}` aliases

* `wasmtime-c-api`: Improve non-support of GC references in `wasm.h` APIs (#8345)

* c-api: Better differentiate between `wasm.h` and `wasmtime.h` APIs

This renames some types and adds some type aliases to help us better distinguish
between `wasm.h` APIs and `wasmtime.h` APIs, primarily for `Store`-related
types. In general, `WasmFoo` is related to `wasm.h` and `WasmtimeFoo` is related
to `wasmtime.h`.

* `StoreRef` -> `WasmStoreRef`
* Introduce the `WasmStore[Data]` and `WasmStoreContext[Mut]` aliases
* `StoreData` -> `WasmtimeStoreData`
* `CStoreContext[Mut]` -> `WasmtimeStoreContext[Mut]`
* Introduce the `Wasmtime{Store,Caller}` aliases

* c-api: Improve non-support of GC references in `wasm.h` APIs

A couple small tweaks: error message improvements, exhaustive matching, etc...

* Fix running wasi-common tests on CI (#8353)

* Fix running wasi-common tests on CI

Turns out we haven't been running wasi-common tests for some time in CI
and they've started failing. Force enable the test at all times and then
fix the test failures. The test failures here were introduced in #8277
and weren't caught due to the test not running and the fix was to relax
the implementation of `fd_pread` to avoid taking multiple mutable
borrows.

* Fix CI

* Update release notes for 20.0.0 (#8358)

A busy release!

* Enable the gc feature by default in the c-api (#8356)

Match the Wasmtime crate in this respect

* wasmtime-c-api: Add support for GC references in `wasmtime.h` APIs (#8346)

Restores support for `externref` in `wasmtime_val_t`, methods for manipulating
them and getting their wrapped host data, and examples/tests for these things.

Additionally adds support for `anyref` in `wasmtime_val_t`, clone/delete methods
similar to those for `externref`, and a few `i31ref`-specific methods. Also adds
C and Rust example / test for working with `anyref`.

* Fix calculation of gc refs in functions (#8355)

* Fix calculation of gc refs in functions

In addition to excluding i31 also exclude funcrefs.

* Review comments

* Remove `wasi_config_preopen_socket` from C header (#8364)

This was removed in #8066

* Tidy up some headers related to shared memory (#8366)

* Tidy up some headers related to shared memory

* Don't declare an anonymous `struct wasmtime_sharedmemory`, instead
  `#include` the actual definition.
* Fix an issue where a header in `sharedmemory.h` referred to a type in
  `extern.h` which wasn't `#include`'d. This function,
  `wasmtime_sharedmemory_into_extern`, additionally isn't necessary as
  it's no different than manually constructing it. Fix this by removing
  this function.

* Run clang-format

* c-api: Fix alignment of `wasmtime_val_*` (#8363)

* c-api: Fix alignment of `wasmtime_val_*`

This commit fixes an issue where `wasmtime_val_raw_t` had an incorrect
alignment. In Rust `ValRaw` contains a `u128` which has an alignment of
16 but in C the representation had a smaller alignment meaning that the
alignment of the two structures was different. This was seen to cause
alignment faults when structure were passed from C++ to Rust, for
example.

This commit changes the Rust representation of `ValRaw`'s `v128` field
to do the same as C which is to use `[u8; 16]`. This avoids the need to
raise the alignment in C which appears to be nontrivial. Cranelift is
appropriately adjusted to understand that loads/stores from `ValRaw` are
no longer aligned. Technically this only applies to the `v128` field but
it's not too bad to apply it to the other fields as well.

* Try alternate syntax for alignof

* Use `--locked` on all `cargo install` in CI, also remove non-locked example (#8369)

* Use `--locked` on all `cargo install` in CI

Prevents any updates to rustc or crates from accidentally causing issues
by ensuring that the same set of deps is used over time.

* Remove rust/WASI markdown parser example

The documentation referring to this example was removed in #6994 and
that forgot to remove this as well. This example is building without a
lock file which is causing issues in #8368.

---------

Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>

.github/actions/install-cargo-vet/action.yml

@@ -16,5 +16,5 @@ runs:
         key: cargo-vet-bin-${{ inputs.version }}
     - run: echo "${{ runner.tool_cache }}/cargo-vet/bin" >> $GITHUB_PATH
       shell: bash
-    - run: cargo install --root ${{ runner.tool_cache }}/cargo-vet --version ${{ inputs.version }} cargo-vet
+    - run: cargo install --root ${{ runner.tool_cache }}/cargo-vet --version ${{ inputs.version }} cargo-vet --locked
       shell: bash

.github/workflows/main.yml

@@ -230,10 +230,9 @@ jobs:
         key: cargo-mdbook-bin-${{ env.CARGO_MDBOOK_VERSION }}
     - run: |
         echo "${{ runner.tool_cache }}/mdbook/bin" >> $GITHUB_PATH
-        cargo install --root ${{ runner.tool_cache }}/mdbook --version ${{ env.CARGO_MDBOOK_VERSION }} mdbook
+        cargo install --root ${{ runner.tool_cache }}/mdbook --version ${{ env.CARGO_MDBOOK_VERSION }} mdbook --locked
     - run: (cd docs && mdbook build)
     - run: cargo build -p wasi-common --features wasmtime/wat,wasmtime/cranelift
-    - run: (cd docs/rust_wasi_markdown_parser && cargo build)
     - run: (cd docs && mdbook test -L ../target/debug/deps)
 
     # Build Rust API documentation.
@@ -360,7 +359,7 @@ jobs:
         path: ${{ runner.tool_cache }}/cargo-ndk
         key: cargo-ndk-bin-${{ env.CARGO_NDK_VERSION }}
     - run: echo "${{ runner.tool_cache }}/cargo-ndk/bin" >> $GITHUB_PATH
-    - run: cargo install --root ${{ runner.tool_cache }}/cargo-ndk --version ${{ env.CARGO_NDK_VERSION }} cargo-ndk
+    - run: cargo install --root ${{ runner.tool_cache }}/cargo-ndk --version ${{ env.CARGO_NDK_VERSION }} cargo-ndk --locked
     - run: cargo ndk -t arm64-v8a check -p wasmtime
 
     # Run clippy configuration
@@ -426,7 +425,7 @@ jobs:
     #
     # Install the OCaml packages necessary for fuzz targets that use the
     # `wasm-spec-interpreter`.
-    - run: cargo install cargo-fuzz --vers "^0.11"
+    - run: cargo install cargo-fuzz --vers "^0.11" --locked
     - run: sudo apt-get update && sudo apt install -y ocaml-nox ocamlbuild ocaml-findlib libzarith-ocaml-dev
     - run: cargo fetch
       working-directory: ./fuzz
@@ -437,7 +436,7 @@ jobs:
     #
     # Afterwards make sure the generated header file is up to date by ensuring
     # that the regeneration process didn't change anything in-tree.
-    - run: cargo install cbindgen --vers "^0.26"
+    - run: cargo install cbindgen --vers "^0.26" --locked
     - run: rustup component add rust-src
     - run: ./build.sh ./embedding/x86_64-unknown-unknown.json
       working-directory: ./examples/min-platform
@@ -747,7 +746,7 @@ jobs:
         path: ${{ runner.tool_cache }}/cargo-nextest
         key: cargo-nextest-bin-${{ env.CARGO_NEXTEST_VERSION }}
     - run: echo "${{ runner.tool_cache }}/cargo-nextest/bin" >> $GITHUB_PATH
-    - run: cargo install --root ${{ runner.tool_cache }}/cargo-nextest --version ${{ env.CARGO_NEXTEST_VERSION }} cargo-nextest
+    - run: cargo install --root ${{ runner.tool_cache }}/cargo-nextest --version ${{ env.CARGO_NEXTEST_VERSION }} cargo-nextest --locked
     - run: |
         cargo miri nextest run -j4 --no-fail-fast \
           -p wasmtime \

RELEASES.md

@@ -6,11 +6,126 @@ Unreleased.
 
 ### Added
 
+* Support for shared WebAssembly memories has been added to the C API.
+  [#7940](https://github.com/bytecodealliance/wasmtime/issues/7940)
+
+* A `Component::component_type` method has been added to reflect on the imports
+  and exports of a component.
+  [#8078](https://github.com/bytecodealliance/wasmtime/issues/8078)
+
+* The `with` key in `bindgen!` now supports remapping entire packages and
+  namespaces.
+  [#8083](https://github.com/bytecodealliance/wasmtime/issues/8083)
+
+* Winch now supports the component model by using Cranelift to generate
+  trampolines.
+  [#8082](https://github.com/bytecodealliance/wasmtime/issues/8082)
+  [#8109](https://github.com/bytecodealliance/wasmtime/issues/8109)
+
+* The WASI-NN backend now supports ONNX.
+  [#7691](https://github.com/bytecodealliance/wasmtime/issues/7691)
+
+* The `wasmtime` CLI now has an `-S inherit-env` flag for inheriting the entire
+  process environment.
+  [#8168](https://github.com/bytecodealliance/wasmtime/issues/8168)
+
+* Winch now supports the WebAssembly memory64 proposal.
+  [#8194](https://github.com/bytecodealliance/wasmtime/issues/8194)
+
+* Embedders can now opt-in to allowing wasmtime-wasi to block the current thread
+  with file operations, for example.
+  [#8190](https://github.com/bytecodealliance/wasmtime/issues/8190)
+
+* A `wasmtime::CodeBuilder` type has been added to refine the configuration of
+  compiling a `Module` or a `Component`.
+  [#8181](https://github.com/bytecodealliance/wasmtime/issues/8181)
+
+* The `wasmtime serve` subcommand now enables configuring preopened directories
+  and environment variables.
+  [#8279](https://github.com/bytecodealliance/wasmtime/issues/8279)
+
 ### Changed
 
+* Support for WebAssembly GC is in the process of being implemented which has
+  required refactoring and reimplementing the existing gc support for
+  `externref`. Many APIs in this area has changed, see linked PRs for more
+  details. Note that the `wasm_*` parts of the C API no longer support
+  `externref`.
+  [#8011](https://github.com/bytecodealliance/wasmtime/issues/8011)
+  [#8196](https://github.com/bytecodealliance/wasmtime/issues/8196)
+
+* The `wasmtime::component::Val` type no longer stores type information and
+  instead must be interpreted in the context of a type.
+  [#8062](https://github.com/bytecodealliance/wasmtime/issues/8062)
+
+* The arguments required for `ResourceAny::try_from_resource` have been
+  simplified by refactoring the internal representation.
+  [#8061](https://github.com/bytecodealliance/wasmtime/issues/8061)
+
+* The arguments required for `wasmtime::component::Linker::func_new` have been
+  simplified by refactoring the internal representation.
+  [#8070](https://github.com/bytecodealliance/wasmtime/issues/8070)
+
+* The pooling allocator is now enabled by default with `wasmtime serve`.
+  [#8073](https://github.com/bytecodealliance/wasmtime/issues/8073)
+
+* The error message for missing imports in has been improved with components.
+  [#7645](https://github.com/bytecodealliance/wasmtime/issues/7645)
+
+* Wasmtime's MSRV is now 1.75.0.
+  [#8205](https://github.com/bytecodealliance/wasmtime/issues/8205)
+
+* Wasmtime's translation of table-related instructions has improved codegen in
+  some common cases, especially with `call_indirect`.
+  [#8063](https://github.com/bytecodealliance/wasmtime/issues/8063)
+  [#8125](https://github.com/bytecodealliance/wasmtime/issues/8125)
+  [#8124](https://github.com/bytecodealliance/wasmtime/issues/8124)
+  [#8134](https://github.com/bytecodealliance/wasmtime/issues/8134)
+  [#8137](https://github.com/bytecodealliance/wasmtime/issues/8137)
+  [#8162](https://github.com/bytecodealliance/wasmtime/issues/8162)
+  [#8159](https://github.com/bytecodealliance/wasmtime/issues/8159)
+  [#8172](https://github.com/bytecodealliance/wasmtime/issues/8172)
+  [#8171](https://github.com/bytecodealliance/wasmtime/issues/8171)
+  [#8139](https://github.com/bytecodealliance/wasmtime/issues/8139)
+  [#8206](https://github.com/bytecodealliance/wasmtime/issues/8206)
+
+* Book-based documentation has been reordered and refactored.
+  [#8130](https://github.com/bytecodealliance/wasmtime/issues/8130)
+
 * The `-S common` flag is renamed to `-S cli`, to better reflect that it provides
   the wasi-cli APIs. `-S common` is still accepted for now, and will be deprecated
   in the future.
+  [#8166](https://github.com/bytecodealliance/wasmtime/issues/8166)
+
+* The tail-call calling convention now supports callee-saved registers to
+  improve performance and allow enabling this WebAssembly proposal by default in
+  the future.
+  [#8246](https://github.com/bytecodealliance/wasmtime/issues/8246)
+
+* The `wasmtime-wasi` crate has been refactored to restructure some items and
+  documentation has been added for most items.
+  [#8228](https://github.com/bytecodealliance/wasmtime/issues/8228)
+
+* Support for the WebAssembly `threads` proposal is now gated by an
+  on-by-default Cargo feature named `threads`.
+  [#8260](https://github.com/bytecodealliance/wasmtime/issues/8260)
+
+* Borrow-checking in `wiggle` has been optimized to not be as fine-grained any
+  more. This is a breaking change if users are relying on the ability to safely
+  mutably borrow disjoint regions of memory.
+  [#8277](https://github.com/bytecodealliance/wasmtime/issues/8277)
+
+### Fixed
+
+* Connection timeouts with `wasi-http` have been fixed.
+  [#8085](https://github.com/bytecodealliance/wasmtime/issues/8085)
+
+* Generating bindings with `bindgen!` now works correctly when some WIT types
+  are not used.
+  [#8065](https://github.com/bytecodealliance/wasmtime/issues/8065)
+
+* Errors in `wasi-http` are no longer accidentally returned as traps.
+  [#8272](https://github.com/bytecodealliance/wasmtime/issues/8272)
 
 * C API bindings now depend on `wasmtime-wasi` instead of `wasi-common`, and the
   `wasi_config_preopen_socket` function is no longer available as a result.

crates/c-api/Cargo.toml

@@ -50,3 +50,4 @@ coredump = ["wasmtime/coredump"]
 addr2line = ["wasmtime/addr2line"]
 demangle = ["wasmtime/demangle"]
 threads = ["wasmtime/threads"]
+gc = ["wasmtime/gc"]

crates/c-api/artifact/Cargo.toml

@@ -31,6 +31,7 @@ default = [
   'addr2line',
   'demangle',
   'threads',
+  'gc',
 ]
 async = ['wasmtime-c-api/async']
 profiling = ["wasmtime-c-api/profiling"]
@@ -44,3 +45,4 @@ addr2line = ["wasmtime-c-api/addr2line"]
 demangle = ["wasmtime-c-api/demangle"]
 wat = ["wasmtime-c-api/wat"]
 threads = ["wasmtime-c-api/threads"]
+gc = ["wasmtime-c-api/gc"]

crates/c-api/include/wasi.h

@@ -168,21 +168,6 @@ WASI_API_EXTERN bool wasi_config_preopen_dir(wasi_config_t *config,
                                              const char *path,
                                              const char *guest_path);
 
-/**
- * \brief Configures a "preopened" listen socket to be available to WASI APIs.
- *
- * By default WASI programs do not have access to open up network sockets on
- * the host. This API can be used to grant WASI programs access to a network
- * socket file descriptor on the host.
- *
- * The fd_num argument is the number of the file descriptor by which it will be
- * known in WASM and the host_port is the IP address and port (e.g.
- * "127.0.0.1:8080") requested to listen on.
- */
-WASI_API_EXTERN bool wasi_config_preopen_socket(wasi_config_t *config,
-                                                uint32_t fd_num,
-                                                const char *host_port);
-
 #undef own
 
 #ifdef __cplusplus

crates/c-api/include/wasmtime/config.h

@@ -164,6 +164,21 @@ WASMTIME_CONFIG_PROP(void, wasm_tail_call, bool)
  */
 WASMTIME_CONFIG_PROP(void, wasm_reference_types, bool)
 
+/**
+ * \brief Configures whether the WebAssembly typed function reference types
+ * proposal is enabled.
+ *
+ * This setting is `false` by default.
+ */
+WASMTIME_CONFIG_PROP(void, wasm_function_references, bool)
+
+/**
+ * \brief Configures whether the WebAssembly GC proposal is enabled.
+ *
+ * This setting is `false` by default.
+ */
+WASMTIME_CONFIG_PROP(void, wasm_gc, bool)
+
 /**
  * \brief Configures whether the WebAssembly SIMD proposal is
  * enabled.

crates/c-api/include/wasmtime/extern.h

@@ -8,6 +8,7 @@
 #define WASMTIME_EXTERN_H
 
 #include <wasmtime/module.h>
+#include <wasmtime/sharedmemory.h>
 #include <wasmtime/store.h>
 
 #ifdef __cplusplus
@@ -89,8 +90,6 @@ typedef uint8_t wasmtime_extern_kind_t;
 /// shared memory
 #define WASMTIME_EXTERN_SHAREDMEMORY 4
 
-struct wasmtime_sharedmemory;
-
 /**
  * \typedef wasmtime_extern_union_t
  * \brief Convenience alias for #wasmtime_extern_union

crates/c-api/include/wasmtime/sharedmemory.h

@@ -56,18 +56,6 @@ wasmtime_sharedmemory_delete(wasmtime_sharedmemory_t *memory);
 WASM_API_EXTERN wasmtime_sharedmemory_t *
 wasmtime_sharedmemory_clone(const wasmtime_sharedmemory_t *memory);
 
-/**
- * \brief Moves shared linear memory into wasmtime_extern_t object
- *
- * \param memory memory to be moved
- * \param out where to store resulting `wasmtime_extern_t`
- *
- * This function moves ownership of `memory` into resulting `wasmtime_extern_t`.
- */
-WASM_API_EXTERN void
-wasmtime_sharedmemory_into_extern(wasmtime_sharedmemory_t *memory,
-                                  wasmtime_extern_t *out);
-
 /**
  * \brief Returns the type of the shared memory specified
  */