[wasmtime CLI]: Expose component interfaces as CLI sub-commands

[tschneidereit]

Feature

Over in #10620, @pchickey proposed stabilizing the format of --invoke. I think that makes sense, but it also caused me to revisit an idea I've had for a long time now: what if we taught wasmtime to generate nice, idiomatic CLIs from WIT exports?

To illustrate the idea, here's a simple example:

For this wit

/// A collection of tools that foo
export interface tools {
  /// A foo function that foos as and bs into strings
  foo: func(a: u32, b: string) -> string;
}

Wasmtime could generate this CLI interface:

$ wasmtime cli.wasm --help
A collection of tools that foo
Usage: cli.wasm <COMMAND>

Commands:
  foo    A foo function that foos as and bs into strings

Options:
  -h, --help     Print help
  -V, --version  Print version

$ wasmtime cli.wasm foo --help
A foo function that foos as and bs into strings

Usage: cli.wasm foo [OPTIONS] [A] [B]

Arguments:
  [A]  A 32-bit integer
  [B]  A string

Options:
...

$ wasmtime cli.wasm foo 42 "or is it?"
The number 42 is very foo, or is it?

Lots of interesting questions around how to handle arg-name: option<ty> (turn into --arg-name?), more complex input types (e.g. accept file names, URLs, and pipes as inputs for streams?), satisfying imports, and others, but I think this would have a lot of potential.

Benefit

This would make it very easy to create simple CLI tools as components. What's more, nothing about these components would be specific to CLIs, necessarily. Instead, the same component could also be used as a building block for creating a web service, or as part of some data processing pipeline, etc.

Implementation

This being an idea more than a plan, I haven't dug into the implementation too much. The biggest issue I can foresee is defining good mappings from various input types (such as "just a string", filenames, URLs, pipes) to component types. Wasmtime would effectively have to act as a user agent, providing mappings from the shell domain into the component model domain.

Alternatives

Do nothing: this isn't some kind of urgent, mission-critical need.

[tschneidereit]

CC @tpmccallum, since you've worked on --invoke support for components, and might find this interesting.

[tpmccallum]

Thanks @tschneidereit - I do find this interesting. This is a great idea.
I will think about this and work to contribute something back in the next week or so.
Chat soon.
Tim

[tpmccallum]

Hi @tschneidereit,
I was just digging into the implementation today and had the following suggestions for discussion.

Config
Perhaps we can add wit-component, wit-parser and wasm-wave to the dependencies:

wit-component = "0.229.0"
wit-parser = "0.229.0"
wasm-wave = "0.229.0"

Parsing
Having wit-component and wit-parser usage at the CLI level should let us interact with types, i.e. FunctionKind, WasmExportKind, description/contents String, etc.

Generate CLI interface
Perhaps this can work both ways, i.e., if I understand your idea. I think that wasmtime can load and parse .wasm, which will allow the user to see details (available exported functions and their arguments/types), etc. Perhaps some helper functions in the src/commands/run.rs, such as:

fn extract_wit_definitions(component: &[u8]) -> Result<(Resolve, World)> {
     // Learn about the component
    // -- implement --
}

fn generate_component_cli(world: &World, resolve: &Resolve) -> Command {
    // Fabricate valid commands that the user can call
    // -- implement --
}

fn parse_component_arg(value: &str, ty: &wit_parser::Type) -> Result<wasmtime::component::Val>{
    // Ensure that we have the component runtime values that are available
    // --implement
}

WAVE
I am not fully across wasm-wave. It looks like WasmTypeKind human-readable WAVE values (for use in the CLI input/output) and Wasmtime Component Val (for machine use inside Rust/Wasmtime runtime) are defined enough to facilitate mappings.

Perhaps using WAVE can facilitate the mappings from the shell domain into the component model domain.

This is a really cool idea that you had. This is just scratching the surface to see if using these tools and this approach is in the right ballpark (and what you intended as part of the implementation). Would love to hear your thoughts. Also super happy to start working on implementing when we align our thoughts a bit more.

Tim

[tschneidereit]

Hey @tpmccallum, I'm really happy you like the idea! And thank you for digging in and thinking about it in more detail.

Perhaps using WAVE can facilitate the mappings from the shell domain into the component model domain.

That is absolutely what I had in mind, yes. I don't think we should necessarily use WAVE exactly as-is and without any additions, but it's a great basis. Additions are certainly needed for things like interpreting different things (such as pipes, paths, and URLs) as input or output streams, and probably more.

Perhaps we can add wit-component, wit-parser and wasm-wave to the dependencies:

These are all already either direct or indirect dependencies for wasmtime, so nothing needs to be added there :)

I think that wasmtime can load and parse .wasm, which will allow the user to see details (available exported functions and their arguments/types), etc. Perhaps some helper functions in the src/commands/run.rs, such as:

I'm not sure if you're proposing the expose these functions and have developers have to explicitly call them to interact with components via the CLI? In any case, my thinking is that all of this should happen under the hood and automatically, without the developer needing to do anything.

[tschneidereit]

One general comment: if we want to pursue this, we should absolutely have an RFC for it: this is big enough and has a sufficiently open design space that we should get alignment on it before potentially wasting a lot of effort on an implementation that people would later disagree with.

That doesn't preclude experimentation, of course, but I do think it should come before trying to get anything into a shippable state.

[tpmccallum]

Hi @tschneidereit,

That all makes sense. Thanks!

I agree that all of this should happen under the hood automatically. I am just experimenting locally, whereby the src/commands/run.rs intercepts the wasmtime command only when:

• a .wasm file is passed in, and
• when the help subcommand (--help/ -h) is present.

The experimental code calls those niche helper functions only when the above conditions are met. It is important to note that the experimental changes only affect passive interactions (parsing/printing), and actual execution/invocation behaviour is not altered (from the current wasmtime run behaviour) in any way.

Agree that an RFC is a great path forward. If my messages are on point with your idea, I would be very happy to create a draft RFC tomorrow as a PR to the rfcs repository. Appreciate you are busy, and this would be my pleasure.

---

Over and above this passive parsing/printing, I think your idea has deeper and more powerful potential. I am trying to get my head around the implications of this automatic parsing/learning and how the idea provides transparency to a level where the coupling is as loose as can be. For example, with this new way of thinking (about the component as a well-defined shared contract), frameworks (like Spin, which executes logic based on triggers, etc.) can now dynamically adapt their behaviour to the component's capabilities on the fly (rather than using explicit hardcoded names and/or custom glue). I would like to still keep thinking about this; very interesting. I digress ...

Am I correct in saying that while there might be more to this idea, the RFC may perhaps just cater for the wasmtime cli.wasm --help and wasmtime cli.wasm foo --help examples. Or do you think we could/should tackle this additional potential in the same RFC?