feat(interface-types) Implement Interface Types (WIT) #787

Hywan · 2019-09-13T13:43:37Z

So far, this PR implements a new crate named wasmer-interface-types. The goal is to implement the WebAssembly Interface Types (WIT) proposal, https://github.com/WebAssembly/interface-types. Keep in mind the proposal is highly unstable and experimental.

The wasmer-interface-types is designed to be runtime agnostic, which means that there is no connection between this crate and wasmer-runtime-core or wasmer-runtime (or any backend crates). The hope is that other runtimes could use wasmer-interface-types according to their needs.

The wasmer-interface-types is composed of 4 parts:

AST: The WIT language, represented as an AST (it's not really abstract though),
Decoders: So far, it contains only one parser that reads the WIT binary language and produces an AST,
Encoders: So far, it contains only one compiler that transforms the AST into "WIT.wat”, i.e. the textual representation of an Interface Types AST
Instructions: WIT defines a new concept called Adapters. An adapter contains a set of instructions. Those instructions are executed when data are bound. So, in more details, this module contains:
- interpreter.rs, a stack-based interpreter, defined by:
  - A compiler that transforms a set of instructions into a set of executable instructions,
  - A stack,
  - A runtime that holds the “invocation inputs” (arguments of the interpreter), the stack, and the Wasm instance (which holds the exports, the imports, the memories, the tables etc.),
- stack.rs, a very light and generic stack implementation, exposing only the API required by the interpreter,
- wasm/*.rs, a set of enums, types, and traits to represent a Wasm runtime —basically this is the part a runtime should take a look to support the wasmer-interface-types crate—.

Progression:

…odule.

…the interpreter.

It requires to create the `wasm::Export` trait, plus the `wasm::Type` and the `wasm::Value` enums.

Also, rename `RepeatWhile` to `RepeatUntil`.

…odule.

MarkMcCaskey

Looks pretty good to me! A lot of my feedback is stylistic but there are a few correctness and other larger concerns here too. Once those get fixed up, I think it should be good to go!

lib/interface-types/src/ast.rs

MarkMcCaskey · 2020-02-12T20:18:07Z

lib/interface-types/src/ast.rs

+        input_types: Vec<InterfaceType>,
+
+        /// The function output types.
+        output_types: Vec<InterfaceType>,


It's worth considering using the structs with the same fields inside this enum if you haven't already. It may not make sense given how it's used -- I haven't read most of the code yet; I think this level of duplication can make sense though

What do you mean by:

It's worth considering using the structs with the same fields inside this enum if you haven't already.

?

Most of the enum's fields are identical to the structs with the same/similar names. It'd be possible to remove most of the fields in the enum and replace it with 1 instance of the correct struct. It's a minor thing and I think there are convincing arguments for both doing it and not doing it.

MarkMcCaskey · 2020-02-12T20:19:13Z

lib/interface-types/src/ast.rs

+    },
+}
+
+/// Represented a forwarded export.


Perhaps the docs should explain what forwarding is

It's not super defined in the spec yet :-D. It should be used to “rename” an export, like main to _start. I'm not sure it will be kept in the future.

lib/interface-types/src/decoders/binary.rs

lib/interface-types/src/encoders/wat.rs

lib/runtime-core/Cargo.toml

lib/interface-types/src/interpreter/instructions/call.rs

lib/interface-types/src/interpreter/wasm/structures.rs

lib/interface-types/src/lib.rs

@MarkMcCaskey

As @MarkMcCaskey noted, `Type` can be corrupted because `field_names` and `field_types` must have the same length. This patch removes the public visibility, and adds methods like `new`, `add_field`, `field_names` and `field_types` to encapsulate `Type` internal data.

The LEB parser is renamed `uleb`. It now checks for overflow, and badly-formed bits, resp. `TooLarge` or `Eof`. More test cases are added, whose from the DWARF 4 standard.

Instead of using `str::from_utf8_unchecked`, this patch updates the code to use `str::from_utf8` and handles the error appropriately.

Hywan · 2020-02-13T13:41:15Z

bors r+

787: feat(interface-types) Implement Interface Types (WIT) r=Hywan a=Hywan So far, this PR implements a new crate named `wasmer-interface-types`. The goal is to implement the WebAssembly Interface Types (WIT) proposal, https://github.com/WebAssembly/interface-types. Keep in mind the proposal is highly unstable and experimental. The `wasmer-interface-types` is designed to be _runtime agnostic_, which means that there is no connection between this crate and `wasmer-runtime-core` or `wasmer-runtime` (or any backend crates). The hope is that other runtimes could use `wasmer-interface-types` according to their needs. The `wasmer-interface-types` is composed of 4 parts: * AST: [The WIT language, represented as an AST](https://github.com/Hywan/wasmer/blob/feat-interface-types/lib/interface-types/src/ast.rs) (it's not really abstract though), * Decoders: So far, it contains only [one parser that reads the WIT binary language](https://github.com/Hywan/wasmer/blob/feat-interface-types/lib/interface-types/src/decoders/binary.rs) and produces an AST, * Encoders: So far, it contains only [one compiler that transforms the AST into "WIT.wat”](https://github.com/Hywan/wasmer/blob/feat-interface-types/lib/interface-types/src/encoders/wat.rs), i.e. the textual representation of an Interface Types AST * Instructions: WIT defines a new concept called Adapters. An adapter contains a set of instructions. Those instructions are executed when data are bound. So, in more details, this module contains: * [`interpreter.rs`](https://github.com/Hywan/wasmer/blob/feat-interface-types/lib/interface-types/src/interpreter/mod.rs), a stack-based interpreter, defined by: * A compiler that transforms a set of instructions into a set of _executable_ instructions, * A stack, * A runtime that holds the “invocation inputs” (arguments of the interpreter), the stack, and the Wasm instance (which holds the exports, the imports, the memories, the tables etc.), * [`stack.rs`](https://github.com/Hywan/wasmer/blob/feat-interface-types/lib/interface-types/src/interpreter/stack.rs), a very light and generic stack implementation, exposing only the API required by the interpreter, * [`wasm/*.rs`](https://github.com/Hywan/wasmer/blob/feat-interface-types/lib/interface-types/src/interpreter/wasm/structures.rs), a set of enums, types, and traits to represent a Wasm runtime —basically this is the part a runtime should take a look to support the `wasmer-interface-types` crate—. Progression: * [x] WIT binary parser, * [x] WIT.wat compiler, * [ ] Interpreter: * [ ] Wasm runtime traits to represent: * [x] Instance, * [x] Export, * [x] Memory, * [x] Import, * [ ] Table, * [x] Generic stack implementation, * [ ] Implemented instructions: * [x] `arg.get`, * [x] `call`, * [x] `call-export`, * [x] `read-utf8`, * [x] `write-utf8`, * [ ] `as-wasm`, * [ ] `as-interface` * [ ] `table-ref-add` * [ ] `table-ref-get` * [ ] `call-method` * [ ] `make-record` * [ ] `get-field` * [ ] `const` * [ ] `fold-seq` * [ ] `add` * [ ] `mem-to-seq` * [ ] `load` * [ ] `seq.new` * [ ] `list.push` * [ ] `repeat-until` * [ ] `seq-to-mem` * [ ] `store` * [ ] Plug `wasmer-interface-types` into `wasmer-runtime-core` Co-authored-by: Ivan Enderlin <ivan.enderlin@hoa-project.net>

bors · 2020-02-13T14:09:46Z

Build succeeded

continuous-integration/travis-ci/push
wasmerio.wasmer

1214: test(interface-types) Add `wasmer-interface-types` to `make check` r=MarkMcCaskey a=Hywan Sequel of #787. Forgot to update the changelog, and to add `wasmer-interface-types` to `make check`. Co-authored-by: Ivan Enderlin <ivan.enderlin@hoa-project.net> Co-authored-by: Mark McCaskey <5770194+MarkMcCaskey@users.noreply.github.com>

Hywan added the 📦 lib-deprecated About the deprecated crates label Sep 13, 2019

Hywan self-assigned this Sep 13, 2019

Hywan added 14 commits September 18, 2019 17:12

feat(wasmer-interface-types) Draft.

b3af77c

feat(interface-types) Continue.

7ca546e

feat(interface-types) Continue.

45ba77c

feat(interface-types) Split into multiple files.

1c1b74b

test(interface-types) Add test case for “unit” parsers.

dc254e0

test(interface-types) Add test cases for higher-level parsers.

24ac7a6

fix(interface-types) Fix visibility of various symbols.

5a8a2b9

feat(interface-types) Move TryFrom from the ast to the binary m…

a7ffffc

…odule.

fix(interface-types) get-field #1 argument is of type InterfaceType.

4ba9aac

feat(interface-types) Draft the WAT encoder.

40613d3

feat(interface-types) Continue the WAT encoder.

6279b3e

feat(interface-types) Add new instructions.

bd3a888

chore(interface-types) Update to Wasmer 0.7.0.

480fe0f

chore(interface-types) Bump to 0.7.0.

6ec35c8

Hywan force-pushed the feat-interface-types branch from a4972cc to 6ec35c8 Compare September 18, 2019 15:15

Hywan added 6 commits September 18, 2019 18:02

feat(interface-types) Encode Interfaces to WAT.

fc9389d

feat(interface-types) Draft instruction interpreter.

dc3c72e

feat(interface-types) Continue.

2f3c37f

feat(interface-types) Continue.

c633450

feat(interface-types) Add an abstract Wasm instance and a runtime to …

62e1f78

…the interpreter.

feat(interface-types) Implement CallExport executable instruction.

9d4c983

It requires to create the `wasm::Export` trait, plus the `wasm::Type` and the `wasm::Value` enums.

Hywan added 🎉 enhancement New feature! 📚 documentation Do you like to read? 🧪 tests I love tests labels Sep 20, 2019

Hywan added 4 commits September 20, 2019 11:55

feat(interface-types) Stack supports Default.

39a8178

chore(interface-types) Fix clippy warnings.

2237e62

feat(interface-types) The interpreter stack contains Wasm values.

b7b37d2

feat(interface-types) Check signature of the exported function to call.

56afb4d

Hywan added 9 commits February 10, 2020 15:44

doc(interface-types) Improve documentation of the ast module.

2fc1fbb

doc(interface-types) Improve documentation of the binary module.

9fda6f9

test(interface-types) Test the parse parser.

98fb069

doc(interface-types) Add a doctest for decoders::binary::parse.

55ae0e4

doc(interface-types) Improve documentation of the wat module.

c5b963b

doc(interface-types) Improve documentation of the macros module.

98c7309

doc(interface-types) Improve documentation of the instruction module.

b3c102d

Also, rename `RepeatWhile` to `RepeatUntil`.

doc(interface-types) Improve the documentation of the instruction m…

6b0e43b

…odule.

doc(interface-types) Improve the documentation of the interpreter m…

13b9cb3

…odule.

MarkMcCaskey approved these changes Feb 12, 2020

View reviewed changes

Hywan added 13 commits February 13, 2020 11:24

doc(interface-types) Improve documentation of decoders::binary::leb.

1ad42d8

feat(decoders) Improve LEB parser in the binary module.

370fd6d

The LEB parser is renamed `uleb`. It now checks for overflow, and badly-formed bits, resp. `TooLarge` or `Eof`. More test cases are added, whose from the DWARF 4 standard.

fix(decoders) Remove the unsafe block in the string parser.

3f0c32b

Instead of using `str::from_utf8_unchecked`, this patch updates the code to use `str::from_utf8` and handles the error appropriately.

chore(decoders) Simplify code.

d8f9230

chore(decoders) Simplify code.

722727b

feat(encoders) Implement ToString instead of From<…> for String.

13cee90

chore(encoders) Changing unimplemented!() to todo!().

6891517

chore(interface-types) Simplify code.

ae6e261

doc(interface-types) Add #[deny(missing_docs)].

9559747

feat(interface-types) Forbids unsafe code.

eb2f9db

fix(interface-types) Clean up cargo features.

7f8d916

Merge branch 'master' into feat-interface-types

c697e68

bors bot merged commit c697e68 into wasmerio:master Feb 13, 2020

Hywan added a commit to Hywan/wasmer that referenced this pull request Feb 13, 2020

doc(changelog) Add wasmerio#787.

2e2c5ab

Hywan mentioned this pull request Feb 13, 2020

test(interface-types) Add wasmer-interface-types to make check #1214

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(interface-types) Implement Interface Types (WIT) #787

feat(interface-types) Implement Interface Types (WIT) #787

Hywan commented Sep 13, 2019 •

edited by d0iasm

Loading

MarkMcCaskey left a comment •

edited

Loading

MarkMcCaskey Feb 12, 2020

Hywan Feb 13, 2020

MarkMcCaskey Feb 13, 2020

MarkMcCaskey Feb 12, 2020

Hywan Feb 13, 2020

Hywan commented Feb 13, 2020

bors bot commented Feb 13, 2020

feat(interface-types) Implement Interface Types (WIT) #787

feat(interface-types) Implement Interface Types (WIT) #787

Conversation

Hywan commented Sep 13, 2019 • edited by d0iasm Loading

MarkMcCaskey left a comment • edited Loading

Choose a reason for hiding this comment

MarkMcCaskey Feb 12, 2020

Choose a reason for hiding this comment

Hywan Feb 13, 2020

Choose a reason for hiding this comment

MarkMcCaskey Feb 13, 2020

Choose a reason for hiding this comment

MarkMcCaskey Feb 12, 2020

Choose a reason for hiding this comment

Hywan Feb 13, 2020

Choose a reason for hiding this comment

Hywan commented Feb 13, 2020

bors bot commented Feb 13, 2020

Build succeeded

Hywan commented Sep 13, 2019 •

edited by d0iasm

Loading

MarkMcCaskey left a comment •

edited

Loading