-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Design] Component Registry Dependencies #42
Comments
I've opened this issue to both document and discuss how best to integrate I'm open to any feedback regarding this design, especially feedback that improves the DX of After a little bike-shedding in this issue, I'll turn this into a PR for proper review. |
It might be worthwhile to update or refer to the glossary over in https://github.com/bytecodealliance/SIG-Registries/blob/main/glossary.md |
Agreed! I think it'd be best to bike shed here a little before pushing a glossary update upstream (if necessary), but I will definitely do so. |
There's certainly plenty to dig in on. Nice write-up! The addition of Could |
Something we can do with components but very few other languages can do, is to (potentially) have the ability to supply multiple versions of a dependency. It might be worth sketching out how we think that will work here (but not necessarily implement it for the MVP). |
I'm not entirely sure I understand; if you mean when implementing component
Does that make sense or did I completely misunderstand your question? |
One part I might bikeshed a bit is the usage of [package.metadata.component]
implements = "fastly.compute-at-edge"
[package.metadata.component.dependencies]
fastly = "0.1"
wasi = "1.0" where the [package.metadata.component]
implements = "my-package.my-world"
[package.metadata.component.dependencies]
fastly = "1.0"
my-package = { path = "wit" } where you would then write:
This would allow avoiding |
I like that as it allows for pointing at someone else's world and using it as is without having to author a
And if they want to define a custom world based on other worlds / interfaces, they'd use the hybrid approach in your example. I'll update the design shortly. |
I'll add that we probably can't support the Perhaps a shorthand of Edit: updated the design to use the shorthand form. |
So I was thinking about this more when walking my dog. The problem with not having a Say for example I'm authoring a component targeting a particular world (e.g. But now I, as the component author, want to make use of another component (which hopefully in a thriving component ecosystem is commonplace). To do so, I now need to know to create a That seems complicated (albeit perhaps automatable by Perhaps, as a possible solution, if the world being implemented isn't from a local path, any dependency on a component package translates to an import merged in with the world being implemented? Is that too magical or no? Another thing to consider is that a local I still think there is something to the clear demarcation of version requirements go in |
I think though component dependencies may work out differently? I think of a world as "I can't work unless you give me this and I give you that" whereas a component dependency is lower level where it's an internal implementation detail that possibly could be bundled within the component itself (or imported for a registry scenario). In that sense would component dependencies necessarily show up in WIT files? |
I don't view component dependencies as internal implementation details when authoring a component at all. From the perspective of the tooling that produces component packages (e.g. It's not the domain of a tool like To me, it's the domain of a composition tool to resolve the abstract (instance) imports to either:
Similarly with exports, it should be possible for a composition tool to specify what gets exported from the resulting component, allowing a target world to be satisfied by an export coming from one internal instance and another export coming from a different internal instance. Composition is where the fun stuff happens. That's the tool that would enforce that the resulting component adheres to a target world, if used, ensuring the resulting component type is a subtype of the world. These composed components could, of course, also be published to a registry, expressing their dependencies on other components as component imports (thereby "locking" the composition to that particular implementation). My understanding of this might be off, but I think we would want to produce component packages from the language tooling as abstract as possible while also expressing dependencies on other components directly (via instance imports). |
To put this all another way, I want the language-specific component tooling to be able to express:
This is mostly why I omitted "worlds" from the design above as it felt like, to me, that this particular tooling isn't as concerned with worlds other than perhaps as a shorthand for explicitly importing and exporting the interfaces specified by a world, e.g.: component {
include wasi.command // in theory a world describing commands
import foo: bar
greet: func(string)
} vs. component {
import wasi-fs: wasi.fs // imported by `wasi.command`
...
execute: func() // in theory whatever exports for `wasi.command`
import foo: bar
greet: func(string)
} In this example, the produced component isn't a subtype of the |
Great writeup and great discussion! First, just as a naming bikeshed, could we call this unified (interface+world) package a "Wit" package (instead of a "definition" package)? My reasoning here is that "definition" is used pretty broadly in core wasm and the component model and in general refers to anything that can be inserted into an index space, thus covering types, functions, instances, etc. Even "components" and "modules" are definitions, so in a sense all packages are "definition" packages. As a second bikeshed, to be consistent with the "targets"/"supports" terminology suggested here and here, perhaps we could say "targets" instead of "implements" in Cargo.toml? In theory it's unambiguous in the context of a component-producer toolchain that when we say "implement" we mean "target", but I was thinking it might be nice to just be consistent in the use of these two terms instead of "implements". Lastly, I agree with Alex that we would ideally stick with But I also agree with Peter that, when I am building a reusable (unlocked) component for publication and reuse, I want to create a component with the variety of kinds of imports that Peter list and let some downstream consumer of this component figure out exact dependency versions, virtualizations, etc when building the final (locked) component I want to execute. My impression of how this works is that, when I'm authoring a component, I start with a "base" world that I'm "targeting", and then I add extra dependencies (on both interfaces and component implementations) in my As a side thought on the interaction between worlds and dependencies: in a normal unlocked component, dependencies on other components appear as |
👍 on simply "wit" package, using "targets" for world terminology when authoring components, and also not using a wit syntax specifically for defining a component type when that's really what a world is. However, it's still not clear to me that we would want to have what world is being targeted in It seems like always having a wit file for the component (optionally pointed at by I'd personally like to delegate the entire world definition to wit and let |
After some discussion with the Registry SIG, I think we'll move forward with the ability to describe the component being authored's world in I think this will strike the right balance between good initial developer experience (i.e. doesn't make |
This commit adds a design document for the developer experience surrounding using packages from a component registry in `cargo-component`. Closes bytecodealliance#42. `cargo-component`
This commit adds a design document for the developer experience surrounding using packages from a component registry in `cargo-component`. Closes bytecodealliance#42.
Thanks everyone for feedback on this. I've put up PR #43 that I hope strikes the right balance between what belongs in |
* Add a design document around registry integration. This commit adds a design document for the developer experience surrounding using packages from a component registry in `cargo-component`. Closes #42. * Clean up TOML examples so they parse for highlighting. * Add more examples. This commit adds examples for exporting things from a world definition. * Remove `file://` URLs for local filesystem registries. * Fix typo. * Update docs/design/registries.md Co-authored-by: Brian <brian.hardock@fermyon.com> * Update docs/design/registries.md Co-authored-by: Brian <brian.hardock@fermyon.com> * Update docs/design/registries.md Co-authored-by: Brian <brian.hardock@fermyon.com> * Update the `targets` field in the component metadata. This commit updates the `targets` field in the component metadata to better represent targeting a local world via a `path` field. As a result, the previous `world` field has been removed. * Update docs/design/registries.md Co-authored-by: Brian <brian.hardock@fermyon.com> --------- Co-authored-by: Brian <brian.hardock@fermyon.com>
Component Registry Dependencies
Overview
Currently
cargo-component
supports implementing a component by specifyingindividual imports and exports defined by local
wit
documents viaCargo.toml
.At the time
cargo-component
was originally implemented, it was imagined thata component registry might store individual interface definitions as packages,
enabling registry dependencies to be specified in a
Cargo.toml
file as:Therefore at most one interface could be defined in a
wit
document and storedin a component registry "interface" package.
As a consequence of this, a
world
stored in a registry would thenneed to explicitly reference each interface being imported and exported from
their individual interface packages; this is certainly not ergonomic and would
definitely contribute to an unnecessary proliferation of packages.
wit
has since evolved to allow multiple interfaces and worlds to be definedin one or more
wit
documents. To facilitate this,wit
now has syntax for using types from other documents.With this more flexible approach to defining interfaces,
cargo-component
needs a new mechanism for specifying dependencies that are stored in a
component registry.
This document proposes a design for specifying dependencies from a component
registry for
cargo-component
.Registry package types
Before discussing the proposed design, it might be useful to discuss some
terminology surrounding the types of packages that might be stored in a
component registry.
Previously, there was discussion around there being three types of packages in
a component registry: an interface package, a world package, and a
component package.
Both interface and world packages are simply WebAssembly components
containing only type information; in terms of the component model proposal,
the former describes an instance type and the latter describes a component
type.
A component package is an implementation of a WebAssembly component; thus it
contains type information and executable code.
With the introduction of the
use
syntax, this proposal suggests reducing thetypes of registry packages to two: a wit package and the
aforementioned concept of a component package.
A wit package, much like the concepts of interface and world
packages, contains only definitions of types, interfaces, and worlds. However,
a wit package may store any number of definitions, including defining
both interfaces and worlds in the same package.
Design overview
Note:
cargo-component
should support sourcing packages from multipleregistries, potentially defaulting to a particular registry instance.
This design proposes that
cargo-component
reads only the versionrequirements for component registry dependencies from
Cargo.toml
.An example
Cargo.toml
of a "greeter" component might look like:The short form of a dependency is
<name> = "<package-id>:<version>"
.The general form is
<name> = { version = "<version>", package = "<package-id>" }
where
name
here maps to a package name used incomponent.wit
(see below)and
package-id
is the qualified identifier of the package in a component registry.Local
wit
documents may still be referenced by using apath
key instead of thepackage
key.In this example,
Cargo.toml
is only specifying that version1.2.3
of the
webassembly/wasi
package and version3.2.1
of themy-org/formatter
package be used.
Note that what is used from the packages is not specified in
Cargo.toml
;thus it doesn't describe the world of the component being built in
any way.
To specify the component's world, a
wit
file with a default name ofcomponent.wit
is used:In this example, the component imports two interfaces: the
fs
interface froma package named
wasi
and the default interface from theformatter
package.The former is used to print the greeting to the console and the latter is used
to format the message based on whatever formatter implementation
is supplied at runtime.
The component will directly export a function named
greet
that willultimately print a greeting for the given name.
Because
cargo-component
resolves dependencies ahead-of-time,wit-parser
only needs to be instructed where to locate the
wasi
andformatter
packages tosuccessfully parse
component.wit
.Implementation
To implement this approach,
cargo-component
will parseCargo.toml
to figureout what component registry dependencies are required.
For consistent builds, it will also consult a lock file (design TBD) for
specific versions and signatures of the dependencies to use.
cargo-component
will contact one or more registries to download (or update)the package logs of the dependencies, verify the logs, and resolve the version
requirements to specific versions to download and cache locally.
It will then parse
./component.wit
(the path can be changed inCargo.toml
, ifdesired) and provide
wit-parser
with the paths to the cached packagedependencies.
From this, the definition of a world will be derived and used to generate the
bindings needed to build the component for commands like
cargo component build
.Finally, the resulting component will encode unique
url
s for its imports andexports based on what packages were resolved (and from where) by
cargo-component
.Benefits
A few benefits to this approach:
Dependencies are specified similar to how they are specified for Rust
crates in
Cargo.toml
: version requirements go inCargo.toml
andwhat is used from the dependencies is specified in "source code"
(
component.wit
in this case).wit-parser
does not need to be made registry-aware; packages areresolved to local definitions prior to its invocation.
Knowledge of a
component.wit
file is transferable to other implementationlanguages: it is just a
wit
document to describe the component beingbuilt and therefore other language-specific tooling would likely also use a
component.wit
file for generating bindings.Tooling for other languages
In addition to Rust, this approach can also be adopted for other
language-specific tooling.
For example, in JavaScript, tooling that wraps
npm
could specify dependencyversion requirements in
package.json
and use acomponent.wit
file tospecify the component type for generating bindings.
Even
wit-bindgen
CLI (or a tool wrapping it) could be made registry-aware bysourcing version requirements from a file (
bindgen.toml
?) and use acomponent.wit
file to generate bindings for supported languages.The text was updated successfully, but these errors were encountered: