CLI Embedding Proposal

[esoterra]

Add a Typed Main proposal to the component model.

Rendered

[lukewagner]

Thanks for writing this up! Quick high-level comment: what if we titled this document "Command-Line Interface Embedding" and named the file CLIEmbedding.md and slightly reframed the intro to say something to the effect of:

• A high-level goal of the component model is to be able to run a single component portably on multiple hosts.
• One such host is a classic Unix-style Command-Line Interface (CLI).
• This document describes how a component can be generically interpreted by a component-enabled CLI to allow components to be directly instantiated and invoked from the command line in an idiomatic manner, corresponding to use case #2 in this section

My thinking here is that, at the component-model level, what we're defining here is symmetric to the JS Embedding (which will likely, as it expands to go into more detail, get factored out into a symmetric JSEmbedding.md), treating the CLI (well, sh/bash/...) as just another programming language client.

What's neat about this context is that it's not WASI-specific. The only WASI interaction here is for how files work, but we could even generalize this part of the embedding to say that it works for any resource type import that is known to the host.

[esoterra]

Thanks, that makes sense to me. I've rendered your suggested changes.

[lukewagner]

Thanks a bunch! Looks generally good to me with these tweaks but I'd be interested to see what other feedback comes up from the upcoming subgroup meeting before final approval/merge.

[lukewagner]

Suggested change

	# Typed Main
	# Command-Line Interface Embedding

[lukewagner]

It seems to me like the long name should always be accepted (since it's explicitly part of the component interface as an import) and that the @flag(short) annotation only adds an additional syntactic sugar option. One can imagine a scenario where I use some general component tooling (say, a component discovery tool which doesn't know about the CLI structured annotation) to look at the signature of the component and I see a value import named foo so I'm unpleasantly surprised when I try to pass --foo on the CLI and it's rejected. (And then if we agree on that, I think there's no need for a @flag(long).)

[ricochet]

I'm leaning towards this answer to always export long but it would be great to hear other opinions. I almost always want a long-named flag when I have a short and it's difficult to come up with reasons for why that would be a problem. On the other hand, the way we would have to back out of this implementation once we standardize it is awkward in case we really do find a use-case where providing a long-named flag is not desired. A contrived example would be for a drop-in replacement of an existing CLI that only wants to accept inputs they have tests for (aka short-names), then we would need to support something like @flag(short, no-long).

[lukewagner]

Yeah, that's a good case to think about. My expectation here is that, for a large number of reasons, this automatic CLI Embedding won't be able to emulate the variety of CLI syntaxes in use for tools today, so we shouldn't worry too much about these exact-emulation scenarios. On the flip side, thinking about this issue afresh, it seems like the "adding CLI annotations should always be backwards-compatible" principle mentioned below is a useful one to preserve.

[lukewagner]

Once @fibonacci1729 posts a sketch .world file syntax (soon, I think), it'd be very nice to update the syntax in all these wit blocks, which would have the nice effect of being clear what's an import and export.

[npmccallum]

@Kylebrown9 Thanks for presenting today. I think this is a good direction overall.

However, I'm really uncomfortable with imports being both positional and named arguments by default. This has several drawbacks:

1. It isn't how people expect CLIs to work in existing programs. I cannot think of a single application that has this behavior.
2. Based on peoples' experience, the behavior that specifying a named argument changes the positional arguments will appear somewhat surprising. Although this is deterministic, it will be unexpected.
3. Because shifting positional arguments in the presence of named arguments will produce unexpected behavior, it will also potentially produce insecure or destructive behavior.

I propose instead the following:

1. Imports may be either named or positional; not both.
2. Imports are automatically treated as named arguments.
3. Developers may use an annotation to mark an import as a positional argument.

This does something useful by default and produces no unexpected behavior.

[lukewagner]

That concern makes sense to me! So then agreed to saying that, by default, all imports are named and you can't pass them positionally from the CLI; you have to opt-in to allowing an import to be supplied positionally with a CLI annotation.

Mirroring my comment above regarding short/long, one property that I think is desirable is: if you can invoke a component successfully from the CLI without the CLI annotations, adding the CLI annotations shouldn't break you; it should only add new available syntax. This seems valuable because it allows you to (1) add CLI annotations to any existing component without worrying that you're breaking someone, and (2) invoke a component from the CLI after looking at the component's type in a type-viewer tool/UI that doesn't understand the CLI annotations (i.e., you should be able to ignore the CLI annotations). Based on this desired property, I'd propose that, when you opt-in to positional syntax, that doesn't disallow you from using the named syntax. My expectation here is that the CLI Embedding can define what constitutes a valid combination of named/positional CLI arguments in an unsurprising and safe manner similar to Python. Does that make sense?

[esoterra]

I'm in agreement with @npmccallum, the combination of positional-or-named arguments and optional arguments creates a situation where it very easy to accidentally bind a positionally passed value to the wrong argument. In the interest of footgun non-proliferation, I'm in favor of going with things being named-only by default.

@lukewagner to make sure I understand you correctly. Are you just asking that when users specify some arguments that are positional they are still able to use other arguments that are named? which is distinct from allowing you to opt into arguments which are both. If so, I'm happy to agree, that is my intention.

[lukewagner]

Not quite: I'm suggesting that, when you opt-in to positional for a particular import (via an annotation), that doesn't disable the ability to name that same particular import. The idea being that annotations should only allow additional syntax, not disable previously-working (before the annotation) syntax.

[lukewagner]

Returning to this with the benefit of seeing how everything else has developed in the meantime: a different way to think about creating a "CLI Embedding" is to consider Bash to be just another programming language that binds to WIT, like JS, Python, Go, Rust, etc. From this perspective, there doesn't need to be a special "CLI Embedding" in the Component Model spec but, rather, Bash toolchain support which can be built separately as with all other languages. A neat consequence of this switch in perspective is that, just like there is both jco transpile and jco componentize (i.e., components-in-a-native-JS-runtime and JS-in-components) which implement the same bindings in two different directions, you could also imagine componentizing Bash (which actually sounds pretty cool).

Based on that understanding, maybe we don't need to develop this idea here, but in some other toolchain repo (which we could link to from here, if anyone starts working on it).

[esoterra]

That makes sense to me. I'll shelve this PR and we can treat this as just another host which doesn't require any specification support.