Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for resources to wit-component #1084

Merged
merged 3 commits into from
Jul 5, 2023

Conversation

alexcrichton
Copy link
Member

This commit fully integrates resource types in the component model with
the wit-component crate, implementing features such as:

  • A WIT package using resource types can now be round-tripped through
    its WebAssembly encoding. This enables use cases such as wit-bindgen
    which embed type information in custom sections of core wasm binaries
    produced by native compilers.

  • Core modules can be lifted into components and the component can use
    resources. This provides a target for wit-bindgen and other code
    generators to use when generating guest code. Resource intrinsics and
    destructors are all available to the core wasm as desired.

  • WIT can be inferred from components using resources, where functions
    are represented as resource-related functions in WIT.

  • The roundtrip-wit fuzzer is extended with resources support meaning
    all of the above support will be fuzzed on OSS-Fuzz.

This required a number of refactorings in wit-component especially
around how type information was handled. Previous processing was a bit
fast-and-loose because where exactly a type was defined didn't really
matter since everything was nominal. With resource types, however,
definition locations are significant and this required some fixes to
previous processing. One example of this is that
WebAssembly/component-model#208 was discovered through this work and the
fixes required were implemented previously and further handled here in
wit-component.

Overall this PR has been almost exclusively fuzz-driven in its
development. I started out with the bare bones of getting simple
components working with resources being imported and exported, then
added fuzzing support to wit-smith, then let the fuzzer go wild. Quite
a few issues were discovered which led to all of the refactorings and
processing here in this PR. I definitely won't claim that this is a
simplification at all to wit-component by any measure. Rather it's
taking a complicated codebase and making it more complicated. In my
mind though the "saving grace" is that I'm pretty confident in the
testing/fuzzing story here. It's relatively easy to isolate issues and
add test cases for the various things that can crop up and the fuzzer
has quite good coverage of all the various paths through
wit-component. All that's to say that this is surely not the "best" or
easiest to understand implementation of resources, but it's intended to
be sufficient for now.

Note: this builds on #1081 and #1083 in the first two commits

@alexcrichton
Copy link
Member Author

cc @silesmo as you're likely interested in this as well

crates/wit-smith/src/lib.rs Outdated Show resolved Hide resolved
crates/wit-component/src/encoding.rs Show resolved Hide resolved
alexcrichton and others added 3 commits July 5, 2023 16:11
This commit fully integrates resource types in the component model with
the `wit-component` crate, implementing features such as:

* A WIT package using `resource` types can now be round-tripped through
  its WebAssembly encoding. This enables use cases such as `wit-bindgen`
  which embed type information in custom sections of core wasm binaries
  produced by native compilers.

* Core modules can be lifted into components and the component can use
  resources. This provides a target for `wit-bindgen` and other code
  generators to use when generating guest code. Resource intrinsics and
  destructors are all available to the core wasm as desired.

* WIT can be inferred from components using resources, where functions
  are represented as `resource`-related functions in WIT.

* The `roundtrip-wit` fuzzer is extended with resources support meaning
  all of the above support will be fuzzed on OSS-Fuzz.

This required a number of refactorings in `wit-component` especially
around how type information was handled. Previous processing was a bit
fast-and-loose because where exactly a type was defined didn't really
matter since everything was nominal. With resource types, however,
definition locations are significant and this required some fixes to
previous processing. One example of this is that
WebAssembly/component-model#208 was discovered through this work and the
fixes required were implemented previously and further handled here in
`wit-component`.

Overall this PR has been almost exclusively fuzz-driven in its
development. I started out with the bare bones of getting simple
components working with resources being imported and exported, then
added fuzzing support to `wit-smith`, then let the fuzzer go wild. Quite
a few issues were discovered which led to all of the refactorings and
processing here in this PR. I definitely won't claim that this is a
simplification at all to `wit-component` by any measure. Rather it's
taking a complicated codebase and making it more complicated. In my
mind though the "saving grace" is that I'm pretty confident in the
testing/fuzzing story here. It's relatively easy to isolate issues and
add test cases for the various things that can crop up and the fuzzer
has quite good coverage of all the various paths through
`wit-component`. All that's to say that this is surely not the "best" or
easiest to understand implementation of resources, but it's intended to
be sufficient for now.
These additions are needed for `wit-bindgen` guest binding generation.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>
@alexcrichton alexcrichton enabled auto-merge (squash) July 5, 2023 23:12
@alexcrichton alexcrichton merged commit 65395c8 into bytecodealliance:main Jul 5, 2023
14 checks passed
@alexcrichton alexcrichton deleted the wit-component-full branch July 6, 2023 04:09
alexcrichton added a commit to alexcrichton/wasm-tools that referenced this pull request Jul 11, 2023
Releases recent changes such as:

* More support for the gc proposal. bytecodealliance#1045 bytecodealliance#1059
* Support for resources throughout most tooling. bytecodealliance#1053 bytecodealliance#1068 bytecodealliance#1070 bytecodealliance#1082
  bytecodealliance#1079 bytecodealliance#1083 bytecodealliance#1084 bytecodealliance#1105 bytecodealliance#1113 bytecodealliance#1116
* Support for `include` in WIT files. bytecodealliance#1054 bytecodealliance#1085 bytecodealliance#1088
* WIT worlds may now be rejected if the same interface can be "reached"
  as both an import and an export simultaneously. bytecodealliance#1081 bytecodealliance#1107
* Support for registry metadata in `wasm-tools metadata`. bytecodealliance#1060
* A new subcommand `wasm-tools component targets`. bytecodealliance#1089
* An `--out-dir` argument is supported on `wasm-tools component wit` to
  print the entire `Resolve` instead of just one package. bytecodealliance#1108
* Miscellaneous bug fixes and improvements. bytecodealliance#1061 bytecodealliance#1065 bytecodealliance#1074 bytecodealliance#1073 bytecodealliance#1078 bytecodealliance#1077
  bytecodealliance#1072 bytecodealliance#1086 bytecodealliance#1091 bytecodealliance#1094 bytecodealliance#1114 bytecodealliance#1106
alexcrichton added a commit to alexcrichton/wasm-tools that referenced this pull request Jul 11, 2023
Releases recent changes such as:

* More support for the gc proposal.
  [bytecodealliance#1045](bytecodealliance#1045)
  [bytecodealliance#1059](bytecodealliance#1059)
* Support for resources throughout most tooling.
  [bytecodealliance#1053](bytecodealliance#1053)
  [bytecodealliance#1068](bytecodealliance#1068)
  [bytecodealliance#1070](bytecodealliance#1070)
  [bytecodealliance#1082](bytecodealliance#1082)
  [bytecodealliance#1079](bytecodealliance#1079)
  [bytecodealliance#1083](bytecodealliance#1083)
  [bytecodealliance#1084](bytecodealliance#1084)
  [bytecodealliance#1105](bytecodealliance#1105)
  [bytecodealliance#1113](bytecodealliance#1113)
  [bytecodealliance#1116](bytecodealliance#1116)
* Support for `include` in WIT files.
  [bytecodealliance#1054](bytecodealliance#1054)
  [bytecodealliance#1085](bytecodealliance#1085)
  [bytecodealliance#1088](bytecodealliance#1088)
* WIT worlds may now be rejected if the same interface can be "reached"
  as both an import and an export simultaneously.
  [bytecodealliance#1081](bytecodealliance#1081)
  [bytecodealliance#1107](bytecodealliance#1107)
* Support for registry metadata in `wasm-tools metadata`.
  [bytecodealliance#1060](bytecodealliance#1060)
* A new subcommand `wasm-tools component targets`.
  [bytecodealliance#1089](bytecodealliance#1089)
* An `--out-dir` argument is supported on `wasm-tools component wit` to
  print the entire `Resolve` instead of just one package.
  [bytecodealliance#1108](bytecodealliance#1108)
* Miscellaneous bug fixes and improvements.
  [bytecodealliance#1061](bytecodealliance#1061)
  [bytecodealliance#1065](bytecodealliance#1065)
  [bytecodealliance#1074](bytecodealliance#1074)
  [bytecodealliance#1073](bytecodealliance#1073)
  [bytecodealliance#1078](bytecodealliance#1078)
  [bytecodealliance#1077](bytecodealliance#1077)
  [bytecodealliance#1072](bytecodealliance#1072)
  [bytecodealliance#1086](bytecodealliance#1086)
  [bytecodealliance#1091](bytecodealliance#1091)
  [bytecodealliance#1094](bytecodealliance#1094)
  [bytecodealliance#1114](bytecodealliance#1114)
  [bytecodealliance#1106](bytecodealliance#1106)
alexcrichton added a commit that referenced this pull request Jul 11, 2023
Releases recent changes such as:

* More support for the gc proposal.
  [#1045](#1045)
  [#1059](#1059)
* Support for resources throughout most tooling.
  [#1053](#1053)
  [#1068](#1068)
  [#1070](#1070)
  [#1082](#1082)
  [#1079](#1079)
  [#1083](#1083)
  [#1084](#1084)
  [#1105](#1105)
  [#1113](#1113)
  [#1116](#1116)
* Support for `include` in WIT files.
  [#1054](#1054)
  [#1085](#1085)
  [#1088](#1088)
* WIT worlds may now be rejected if the same interface can be "reached"
  as both an import and an export simultaneously.
  [#1081](#1081)
  [#1107](#1107)
* Support for registry metadata in `wasm-tools metadata`.
  [#1060](#1060)
* A new subcommand `wasm-tools component targets`.
  [#1089](#1089)
* An `--out-dir` argument is supported on `wasm-tools component wit` to
  print the entire `Resolve` instead of just one package.
  [#1108](#1108)
* Miscellaneous bug fixes and improvements.
  [#1061](#1061)
  [#1065](#1065)
  [#1074](#1074)
  [#1073](#1073)
  [#1078](#1078)
  [#1077](#1077)
  [#1072](#1072)
  [#1086](#1086)
  [#1091](#1091)
  [#1094](#1094)
  [#1114](#1114)
  [#1106](#1106)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants