Feature Request: Extensible CLI and scaffolding plugins

[joelanford]

Overview

I would like for Kubebuilder to become more extensible, such that it could be imported and used as a library in other projects. Specifically, I'm looking for a way to use Kubebuilder's existing CLI and scaffolding for Go projects, but to also be able to augment Kubebuilder's project versions with other custom project versions so that I can support the Kubebuilder workflow with non-Go operators (e.g. operator-sdk's Ansible and Helm-based operators).

The idea is for Kubebuilder to define one or more plugin interfaces that can be used to drive what the init, create api and create webhooks subcommands do and to add a new cli package that other projects can use to integrate out-of-tree plugins with the Kubebuilder CLI in their own projects.

Related issues and PRs

#1148
#1171
Possibly #1218

Prototype implementation

https://github.com/joelanford/kubebuilder-exp

Plugin interfaces

Required

Each plugin would minimally be required to implement the Plugin interface.

Plugin interface

type Plugin interface {
    // Version is the project version that this plugin implements.
    // For example, Kubebuilder's Go v2 plugin implementation would return "2"
    Version() string
}

Optional

Next, a plugin could optionally implement further interfaces to declare its support for specific Kubebuilder subcommands. For example:

• InitPlugin - to initialize new projects
• CreateAPIPlugin - to create APIs (and possibly controllers) for existing projects
• CreateWebhookPlugin - to create webhooks for existing projects

Each of these interfaces would follow the same pattern (see the InitPlugin interface example below).

Example InitPlugin interface

type InitPlugin interface {
    Plugin

    // InitDescription returns a description of what this plugin initializes
    // for a new project. It is used to display help.
    InitDescription() string

    // InitHelp returns one or more examples of the command-line usage
    // of this plugin's project initialization support. It is used to display help.
    InitExample() string

    // BindInitFlags binds the plugin's init flags to the CLI. This allows each
    // plugin to define its own command line flags for the `kubebuilder init`
    // subcommand.
    BindInitFlags(fs *pflag.FlagSet)

    // Init initializes a project.
    Init() error
}

Deprecated Plugins

To generically support deprecated project versions, we could also add a Deprecated interface that the CLI could use to decide when to print deprecation warnings:

Deprecated interface

// Deprecated is an interface that, if implemented, informs the CLI
// that the plugin is deprecated.  The CLI uses this to print deprecation
// warnings when the plugin is in use.
type Deprecated interface {
    // DeprecationWarning returns a deprecation message that callers 
    // can use to warn users of deprecations
    DeprecationWarning() string
}

CLI

To make the above plugin system extensible and usable by other projects, we could add a new CLI package that Kubebuilder (and other projects) could use as their entrypoint.

Example Kubebuilder main.go

func main() {
	c, err := cli.New(
		cli.WithPlugins(
			&golangv1.Plugin{},
			&golangv2.Plugin{},
		),
	)
	if err != nil {
		log.Fatal(err)
	}
	if err := c.Run(); err != nil {
		log.Fatal(err)
	}
}

Example Operator SDK main.go

func main() {
	c, err := cli.New(
		cli.WithCommandName("operator-sdk"),
		cli.WithDefaultProjectVersion("2"),
		cli.WithExtraCommands(newCustomCobraCmd()),
		cli.WithPlugins(
			&golangv1.Plugin{},
			&golangv2.Plugin{},
			&helmv1.Plugin{},
			&ansiblev1.Plugin{},
		),
	)
	if err != nil {
		log.Fatal(err)
	}
	if err := c.Run(); err != nil {
		log.Fatal(err)
	}
}

Comments & Questions

Cobra Commands

As discussed earlier as part of #1148, one goal is to eliminate the use of cobra.Command in the exported API of Kubebuilder since that is considered an internal implementation detail.

However, at some point, projects that make use of this extensibility will likely want to integrate their own subcommands. In this proposal, cli.WithExtraCommands() DOES expose cobra.Command to allow callers to pass their own subcommands to the CLI.

In #1148, callers would use Kubebuilder's cobra commands to build their CLI. Here, control of the CLI is retained by Kubebuilder, and callers pass their subcommands to Kubebuilder. This has several benefits:

1. Kubebuilder's CLI subcommands are never exposed except via the explicit plugin interface. This allows the Kubebuilder project to re-implement its subcommand internals without worrying about backwards compatibility of consumers of Kubebuilder's CLI.
2. If desired, Kubebuilder could ensure that extra subcommands do not overwrite/reuse the existing Kubebuilder subcommand names. For example, only Kubebuilder gets to define the init subcommand
3. The overall binary's help handling is self-contained in Kubebuilder's CLI. Callers don't have to figure out how to have a cohesive help output between the Kubebuilder CLI and their own custom subcommands.

With all of that said, even this exposure of cobra.Command could be problematic. If Kubebuilder decides in the future to transition to a different CLI framework (or to roll its own) it has to either continue maintaining support for these extra cobra commands passed into it, or it was to break the CLI API.

Are there other ideas for how to handle the following requirements?

• Eliminate use of cobra in CLI interface
• Allow other projects to have custom subcommands
• Support cohesive help output

Other

1. Should the InitPlugin interface methods be required of all plugins?
2. Any other approaches or ideas?
3. Anything I didn't cover that could use more explanation?

/kind feature

[joelanford]

/assign @DirectXMan12
/cc @estroz @hasbro17 @shawn-hurley @camilamacedo86

[DirectXMan12]

Haven't reviewed the actual prototype, will do that shortly. Initial thoughts:

1. Can you put this in a design doc in a PR? Slightly easier to comment on different sections with context (you can literally just copy-and-paste the the content from this issue into the design doc -- we don't enforce any particular structure besides having an example, which you already have :-) )

2. How do you specify init plugins (e.g. --project-version v2:osdk,add-ascii-art)? I imagine that some of them want to persist there plugin-ness to PROJECT, so being able to add config sections to PROJECT and to remember that they're enabled would be nice.

I'm thinking along the lines of addon operator or OSDK, where you might want to automatically invoke the OSDK api plugin when a project was init-ed with the OSDK init plugin.

3. Don't let this derail the discussion, but: can we have some way to just turn config blocks into CLI args, and then we can have a single point of config for args vs PROJECT, and not depend on pflag externally? This might be too much of a "rewrite the world" though. e.g.

type OSDKConfig struct {
ImageVersion string `cli:"img-version"`
}
// becomes --osdk-img-version or something
// maybe have a validate method too

4. Do plugins still get the world state somehow?

[joelanford]

Closing in favor of #1250 to make review and commenting easier.