Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: update Rust and WIT deps #1146

Merged
merged 6 commits into from
Dec 13, 2023
Merged

build: update Rust and WIT deps #1146

merged 6 commits into from
Dec 13, 2023

Conversation

ricochet
Copy link
Contributor

@ricochet ricochet commented Dec 7, 2023

Feature or Problem

We will have to update the adapters shortly after this PR due to bytecodealliance/wasmtime#7663

Note this uses a release branch as 16 is not out yet.

Related Issues

Note that rc-12-05 is piecemeal. Only wasi-http and wasi-cli have a newer release candidate.

Background: WebAssembly/wasi-io#63

Release Information

next

Consumer Impact

This breaks existing WIT definitions against wasi-http.

Manual Verification

cargo test

@ricochet ricochet requested review from a team as code owners December 7, 2023 16:39
Copy link
Contributor

@thomastaylor312 thomastaylor312 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am ok with this, just one concern about pinning to a branch

Cargo.toml Show resolved Hide resolved
Copy link
Member

@rvolosatovs rvolosatovs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks like we'll need to update to Rust 1.74 to use bytecodealliance/wasmtime#7561, I'll work on that in parallel

Cargo.toml Outdated Show resolved Hide resolved
Cargo.toml Outdated Show resolved Hide resolved
@rvolosatovs
Copy link
Member

@ricochet since #1143 has been merged, rebasing this PR on latest main should fix the lint errors and allow us to build latest wasmtime

@rvolosatovs
Copy link
Member

It appears that Wasmtime's incompatibility with cargo vendor strikes again, I'll take a better look tomorrow.
Refs ipetkov/crane#480

@rvolosatovs
Copy link
Member

@ricochet I've made a bit more progress here:

diff --git a/Cargo.toml b/Cargo.toml
index 853b0ea9bd..42fb309538 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -98,10 +98,10 @@ env_logger = { version = "0.10", default-features = false }
 futures = { version = "0.3", default-features = false }
 heck = { version = "0.4", default-features = false }
 hex = { version = "0.4", default-features = false }
-http = { version = "1.0", default-features = false }
+http = { version = "1", default-features = false, features = ["std"] }
 http-body = { version = "1", default-features = false }
 http-body-util = { version = "0.1", default-features = false }
-humantime = { version = "2.1", default-features = false }
+humantime = { version = "2", default-features = false }
 hyper = { version = "0.14", default-features = false }
 ignore = { version = "0.4", default-features = false }
 indicatif = { version = "0.17", default-features = false }
diff --git a/crates/providers/Cargo.lock b/crates/providers/Cargo.lock
index fd0d4211dc..d33f94e02f 100644
--- a/crates/providers/Cargo.lock
+++ b/crates/providers/Cargo.lock
@@ -2603,7 +2603,7 @@ name = "wasmcloud-provider-httpclient"
 version = "0.8.0"
 dependencies = [
  "async-trait",
- "http 1.0.0",
+ "http 0.2.11",
  "reqwest",
  "serde",
  "serde_bytes",
@@ -2623,7 +2623,7 @@ dependencies = [
  "dashmap",
  "flume",
  "futures",
- "http 1.0.0",
+ "http 0.2.11",
  "serde",
  "serde_bytes",
  "serde_json",
@@ -2764,7 +2764,7 @@ dependencies = [
 [[package]]
 name = "wasmtime-wit-bindgen"
 version = "16.0.0"
-source = "git+https://github.com/bytecodealliance/wasmtime?branch=release-16.0.0#759aa585496adfbe47aefb4fdf9608aabf8d64ac"
+source = "git+https://github.com/bytecodealliance/wasmtime?branch=release-16.0.0#64c2a247edd6bfc41bcfccb61848077c116bd9d6"
 dependencies = [
  "anyhow",
  "heck",
diff --git a/crates/providers/Cargo.toml b/crates/providers/Cargo.toml
index 95822f67b0..e4ceced4dd 100644
--- a/crates/providers/Cargo.toml
+++ b/crates/providers/Cargo.toml
@@ -30,7 +30,7 @@ bytes = { version = "1", default-features = false }
 dashmap = { version = "5", default-features = false }
 flume = { version = "0.11", default-features = false }
 futures = { version = "0.3", default-features = false }
-http = { version = "1.0", default-features = false }
+http = { version = "0.2", default-features = false }
 opentelemetry = { version = "0.20", default-features = false }
 opentelemetry-nats = { path = "../opentelemetry-nats" }
 path-clean = { version = "1", default-features = false }
diff --git a/flake.lock b/flake.lock
index 06ec391a23..b6033e32b9 100644
--- a/flake.lock
+++ b/flake.lock
@@ -21,17 +21,17 @@
         "nixpkgs": "nixpkgs"
       },
       "locked": {
-        "lastModified": 1699217310,
-        "narHash": "sha256-xpW3VFUG7yE6UE6Wl0dhqencuENSkV7qpnpe9I8VbPw=",
-        "owner": "ipetkov",
+        "lastModified": 1702471460,
+        "narHash": "sha256-V02yXoDbZEi4uj8e7Wt/YkyaMuZY/Ps3MHpfB+YtwE8=",
+        "owner": "rvolosatovs",
         "repo": "crane",
-        "rev": "d535642bbe6f377077f7c23f0febb78b1463f449",
+        "rev": "8e234a879edc56f828f467fa0dc5502e8498cf95",
         "type": "github"
       },
       "original": {
-        "owner": "ipetkov",
-        "ref": "v0.15.0",
+        "owner": "rvolosatovs",
         "repo": "crane",
+        "rev": "8e234a879edc56f828f467fa0dc5502e8498cf95",
         "type": "github"
       }
     },
@@ -289,7 +289,9 @@
     "nixify": {
       "inputs": {
         "advisory-db": "advisory-db",
-        "crane": "crane",
+        "crane": [
+          "crane"
+        ],
         "fenix": "fenix",
         "flake-utils": "flake-utils",
         "macos-sdk": "macos-sdk",
@@ -388,11 +390,11 @@
     },
     "nixpkgs": {
       "locked": {
-        "lastModified": 1698931758,
-        "narHash": "sha256-pwl9xS9JFMXXR1lUP/QOqO9hiZKukEcVUU1A0DKQwi4=",
+        "lastModified": 1701432845,
+        "narHash": "sha256-06sd2rQ+DPMSueh+hW4MiXbpMSdhQHJOi/sw0vuwqvs=",
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "b644d97bda6dae837d577e28383c10aa51e5e2d2",
+        "rev": "77da99a144cd341408308e0a37622f5edcc6c5ba",
         "type": "github"
       },
       "original": {
@@ -468,6 +470,7 @@
     },
     "root": {
       "inputs": {
+        "crane": "crane",
         "nixify": "nixify",
         "nixlib": "nixlib_3",
         "wasmcloud-component-adapters": "wasmcloud-component-adapters",
diff --git a/flake.nix b/flake.nix
index 6cb58c2d86..f1d5c72471 100644
--- a/flake.nix
+++ b/flake.nix
@@ -16,6 +16,8 @@
     "cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g="
   ];
 
+  inputs.crane.url = "github:rvolosatovs/crane/8e234a879edc56f828f467fa0dc5502e8498cf95";
+  inputs.nixify.inputs.crane.follows = "crane";
   inputs.nixify.inputs.nixlib.follows = "nixlib";
   inputs.nixify.url = "github:rvolosatovs/nixify";
   inputs.nixlib.url = "github:nix-community/nixpkgs.lib";
diff --git a/tests/actors/rust/builtins-component-reactor/wit/deps/cli/command.wit b/tests/actors/rust/builtins-component-reactor/wit/deps/cli/command.wit
index 74811d3272..cc82ae5dc5 100644
--- a/tests/actors/rust/builtins-component-reactor/wit/deps/cli/command.wit
+++ b/tests/actors/rust/builtins-component-reactor/wit/deps/cli/command.wit
@@ -1,7 +1,7 @@
-package wasi:cli@0.2.0-rc-2023-11-10;
+package wasi:cli@0.2.0-rc-2023-12-05;
 
 world command {
-  include reactor;
+  include imports;
 
   export run;
 }
diff --git a/tests/actors/rust/builtins-component-reactor/wit/deps/cli/imports.wit b/tests/actors/rust/builtins-component-reactor/wit/deps/cli/imports.wit
new file mode 100644
index 0000000000..9965ea35ec
--- /dev/null
+++ b/tests/actors/rust/builtins-component-reactor/wit/deps/cli/imports.wit
@@ -0,0 +1,20 @@
+package wasi:cli@0.2.0-rc-2023-12-05;
+
+world imports {
+  include wasi:clocks/imports@0.2.0-rc-2023-11-10;
+  include wasi:filesystem/imports@0.2.0-rc-2023-11-10;
+  include wasi:sockets/imports@0.2.0-rc-2023-11-10;
+  include wasi:random/imports@0.2.0-rc-2023-11-10;
+  include wasi:io/imports@0.2.0-rc-2023-11-10;
+
+  import environment;
+  import exit;
+  import stdin;
+  import stdout;
+  import stderr;
+  import terminal-input;
+  import terminal-output;
+  import terminal-stdin;
+  import terminal-stdout;
+  import terminal-stderr;
+}
diff --git a/tests/actors/rust/builtins-component-reactor/wit/deps/cli/reactor.wit b/tests/actors/rust/builtins-component-reactor/wit/deps/cli/reactor.wit
deleted file mode 100644
index eafa2fd499..0000000000
--- a/tests/actors/rust/builtins-component-reactor/wit/deps/cli/reactor.wit
+++ /dev/null
@@ -1,31 +0,0 @@
-package wasi:cli@0.2.0-rc-2023-11-10;
-
-world reactor {
-  import wasi:clocks/wall-clock@0.2.0-rc-2023-11-10;
-  import wasi:clocks/monotonic-clock@0.2.0-rc-2023-11-10;
-  import wasi:filesystem/types@0.2.0-rc-2023-11-10;
-  import wasi:filesystem/preopens@0.2.0-rc-2023-11-10;
-  import wasi:sockets/instance-network@0.2.0-rc-2023-11-10;
-  import wasi:sockets/ip-name-lookup@0.2.0-rc-2023-11-10;
-  import wasi:sockets/network@0.2.0-rc-2023-11-10;
-  import wasi:sockets/tcp-create-socket@0.2.0-rc-2023-11-10;
-  import wasi:sockets/tcp@0.2.0-rc-2023-11-10;
-  import wasi:sockets/udp-create-socket@0.2.0-rc-2023-11-10;
-  import wasi:sockets/udp@0.2.0-rc-2023-11-10;
-  import wasi:random/random@0.2.0-rc-2023-11-10;
-  import wasi:random/insecure@0.2.0-rc-2023-11-10;
-  import wasi:random/insecure-seed@0.2.0-rc-2023-11-10;
-  import wasi:io/poll@0.2.0-rc-2023-11-10;
-  import wasi:io/streams@0.2.0-rc-2023-11-10;
-
-  import environment;
-  import exit;
-  import stdin;
-  import stdout;
-  import stderr;
-  import terminal-input;
-  import terminal-output;
-  import terminal-stdin;
-  import terminal-stdout;
-  import terminal-stderr;
-}
diff --git a/tests/actors/rust/builtins-component-reactor/wit/deps/http/proxy.wit b/tests/actors/rust/builtins-component-reactor/wit/deps/http/proxy.wit
index 453f590511..0f466c93c1 100644
--- a/tests/actors/rust/builtins-component-reactor/wit/deps/http/proxy.wit
+++ b/tests/actors/rust/builtins-component-reactor/wit/deps/http/proxy.wit
@@ -1,4 +1,4 @@
-package wasi:http@0.2.0-rc-2023-11-10;
+package wasi:http@0.2.0-rc-2023-12-05;
 
 /// The `wasi:http/proxy` world captures a widely-implementable intersection of
 /// hosts that includes HTTP forward and reverse proxies. Components targeting
@@ -6,20 +6,19 @@ package wasi:http@0.2.0-rc-2023-11-10;
 /// outgoing HTTP requests.
 world proxy {
   /// HTTP proxies have access to time and randomness.
-  import wasi:clocks/wall-clock@0.2.0-rc-2023-11-10;
-  import wasi:clocks/monotonic-clock@0.2.0-rc-2023-11-10;
+  include wasi:clocks/imports@0.2.0-rc-2023-11-10;
   import wasi:random/random@0.2.0-rc-2023-11-10;
 
   /// Proxies have standard output and error streams which are expected to
   /// terminate in a developer-facing console provided by the host.
-  import wasi:cli/stdout@0.2.0-rc-2023-11-10;
-  import wasi:cli/stderr@0.2.0-rc-2023-11-10;
+  import wasi:cli/stdout@0.2.0-rc-2023-12-05;
+  import wasi:cli/stderr@0.2.0-rc-2023-12-05;
 
   /// TODO: this is a temporary workaround until component tooling is able to
   /// gracefully handle the absence of stdin. Hosts must return an eof stream
   /// for this import, which is what wasi-libc + tooling will do automatically
   /// when this import is properly removed.
-  import wasi:cli/stdin@0.2.0-rc-2023-11-10;
+  import wasi:cli/stdin@0.2.0-rc-2023-12-05;
 
   /// This is the default handler to use when user code simply wants to make an
   /// HTTP request (e.g., via `fetch()`).
diff --git a/tests/actors/rust/builtins-component-reactor/wit/deps/http/types.wit b/tests/actors/rust/builtins-component-reactor/wit/deps/http/types.wit
index 1dd4214cd4..0f698e769e 100644
--- a/tests/actors/rust/builtins-component-reactor/wit/deps/http/types.wit
+++ b/tests/actors/rust/builtins-component-reactor/wit/deps/http/types.wit
@@ -169,9 +169,16 @@ interface types {
       entries: list<tuple<field-key,field-value>>
     ) -> result<fields, header-error>;
 
-    /// Get all of the values corresponding to a key.
+    /// Get all of the values corresponding to a key. If the key is not present
+    /// in this `fields`, an empty list is returned. However, if the key is
+    /// present but empty, this is represented by a list with one or more
+    /// empty field-values present.
     get: func(name: field-key) -> list<field-value>;
 
+    /// Returns `true` when the key is present in this `fields`. If the key is
+    /// syntactically invalid, `false` is returned.
+    has: func(name: field-key) -> bool;
+
     /// Set all of the values for a key. Clears any existing values for that
     /// key, if they have been set.
     ///
@@ -191,7 +198,7 @@ interface types {
     append: func(name: field-key, value: field-value) -> result<_, header-error>;
 
     /// Retrieve the full set of keys and values in the Fields. Like the
-    /// constructor, the list represents each key-value pair. 
+    /// constructor, the list represents each key-value pair.
     ///
     /// The outer list represents each key-value pair in the Fields. Keys
     /// which have multiple values are represented by multiple entries in this
@@ -308,9 +315,9 @@ interface types {
     headers: func() -> headers;
   }
 
-  /// Parameters for making an HTTP Request. Each of these parameters is an
-  /// optional timeout, with the unit in milliseconds, applicable to the
-  /// transport layer of the HTTP protocol.
+  /// Parameters for making an HTTP Request. Each of these parameters is
+  /// currently an optional timeout applicable to the transport layer of the
+  /// HTTP protocol.
   ///
   /// These timeouts are separate from any the user may use to bound a
   /// blocking call to `wasi:io/poll.poll`.
@@ -319,27 +326,27 @@ interface types {
     constructor();
 
     /// The timeout for the initial connect to the HTTP Server.
-    connect-timeout-ms: func() -> option<duration>;
+    connect-timeout: func() -> option<duration>;
 
     /// Set the timeout for the initial connect to the HTTP Server. An error
     /// return value indicates that this timeout is not supported.
-    set-connect-timeout-ms: func(ms: option<duration>) -> result;
+    set-connect-timeout: func(duration: option<duration>) -> result;
 
     /// The timeout for receiving the first byte of the Response body.
-    first-byte-timeout-ms: func() -> option<duration>;
+    first-byte-timeout: func() -> option<duration>;
 
     /// Set the timeout for receiving the first byte of the Response body. An
     /// error return value indicates that this timeout is not supported.
-    set-first-byte-timeout-ms: func(ms: option<duration>) -> result;
+    set-first-byte-timeout: func(duration: option<duration>) -> result;
 
     /// The timeout for receiving subsequent chunks of bytes in the Response
     /// body stream.
-    between-bytes-timeout-ms: func() -> option<duration>;
+    between-bytes-timeout: func() -> option<duration>;
 
     /// Set the timeout for receiving subsequent chunks of bytes in the Response
     /// body stream. An error return value indicates that this timeout is not
     /// supported.
-    set-between-bytes-timeout-ms: func(ms: option<duration>) -> result;
+    set-between-bytes-timeout: func(duration: option<duration>) -> result;
   }
 
   /// Represents the ability to send an HTTP Response.
@@ -437,16 +444,20 @@ interface types {
     /// The outer `option` represents future readiness. Users can wait on this
     /// `option` to become `some` using the `subscribe` method.
     ///
-    /// The `result` represents that either the HTTP Request or Response body,
-    /// as well as any trailers, were received successfully, or that an error
-    /// occured receiving them. The optional `trailers` indicates whether or not
-    /// trailers were present in the body.
+    /// The outer `result` is used to retrieve the trailers or error at most
+    /// once. It will be success on the first call in which the outer option
+    /// is `some`, and error on subsequent calls.
+    ///
+    /// The inner `result` represents that either the HTTP Request or Response
+    /// body, as well as any trailers, were received successfully, or that an
+    /// error occured receiving them. The optional `trailers` indicates whether
+    /// or not trailers were present in the body.
     ///
     /// When some `trailers` are returned by this method, the `trailers`
     /// resource is immutable, and a child. Use of the `set`, `append`, or
     /// `delete` methods will return an error, and the resource must be
     /// dropped before the parent `future-trailers` is dropped.
-    get: func() -> option<result<option<trailers>, error-code>>;
+    get: func() -> option<result<result<option<trailers>, error-code>>>;
   }
 
   /// Represents an outgoing HTTP Response.

It still does not compile though and it appears that this is currently blocked on wascap bump and we'll have to figure out what IO package to use, since it has changed

ricochet and others added 4 commits December 13, 2023 20:35
Note this uses a release branch as 16 is not out yet.

Co-authored-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Bailey Hayes <behayes2@gmail.com>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Fixup 23034fe, which committed a `Cargo.lock`, which should not be
there

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Fixup 23034fe

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
@rvolosatovs rvolosatovs changed the title feat(runtime): wasmtime 16 build: update Rust and WIT deps Dec 13, 2023
@rvolosatovs rvolosatovs enabled auto-merge (rebase) December 13, 2023 19:38
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Brooks Townsend <brooksmtownsend@gmail.com>
@rvolosatovs rvolosatovs merged commit 7fac3db into main Dec 13, 2023
32 checks passed
@rvolosatovs rvolosatovs deleted the wasmtime-16 branch December 13, 2023 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants