Support auto configuring code generators with meta code-generator cmd

[pwittrock]

User story: As a contributor, I would like to run the set of of supported code-generators against a project following the Kubernetes code structure and conventions without having to manually configure each of them and understand each of their specific semantics.

Background: The code-generator repo has a dozen different code generator commands. Each of the code generator commands takes a different set of flags derived from the directories present in the project. Configure each code generator for every new project or API extension should not be necessary. Instead, we should automatically configure each code generator based which directories are present.

Proof of concept implemented here

[munnerz]

Similar to my comment in #15, have you seen the code-generator scripts in the root of this repo, recently merged? These take care of running the generators for you and configuring them correctly: https://github.com/kubernetes/code-generator/blob/master/generate-groups.sh

(great work with apiserver-builder by the way!)

[pwittrock]

Thanks!

@munnerz where does the script compile the code from? does it check it out?

Also, the script does not seem to run a number of the code generators - openapi, protobuf, etc. Are these run by another script?

[munnerz]

The idea generally is to vendor the entire code-generator repository, and it runs go install to install them to your GOBIN: https://github.com/kubernetes/code-generator/blob/master/generate-groups.sh#L49

There isn't another script for those generators at the moment (or support for it in the existing one). That is definitely something to look into 😄

/cc @sttts

[munnerz]

I think there was previously discussion about making this into a golang app actually. It was decided to put it into a script for the time being (as opposed to an initial version of the PR which was a bash library) to make the transition to a full CLI tool easier.

[sttts]

This is related to kubernetes/kubernetes#53524, maybe even is part of it.

[sttts]

Also, the script does not seem to run a number of the code generators - openapi, protobuf, etc. Are these run by another script?

It's not complete, suitable for sample-apiserver mainly. And as written in kubernetes/kubernetes#53524 this is an intermediate step before we have a better solution, e.g. a golang based tool.

[pwittrock]

@sttts Glad to hear it. Thanks for the context. I may have someone who can work on really building upon this to produce a consumer centric solution - including a golang script and all of the accoutrements (good built in help documentation w/ examples, error checking and messaging, packaging of artifacts, etc). If we put together a PR that took the existing script evolved it, would you be willing to find a reviewer? What would you like to see in terms of a design proposal?

I have a number of items for polish / chrome we could apply to really make the the experience magical. Some examples:

• provide a golang (cobra) binary including help documentation, error checking, and maybe some porcelain for generating boilerplate.go.txt files for common licenses
• provide an optional Bazel rule that could be added to a BUILD file and would automatically take care of downloading generators, installing them, and running them. See this gazelle example for what this might look like
• provide tools to detect changes in inputs, and only running code generators against input files that have changed when running generators (faster iteration)
• provide tools for managing generated files - e.g. clean (delete all generated), add to .gitignore, etc
• do linting and detect common errors at code generation time instead of at compile time

[pwittrock]

@sttts Also, should I go ahead and close this and the other issue then, and put them under the issue you referenced?

[sttts]

@pwittrock sounds really great! Reviewing shouldn't be a big problem. I am happy to support with that myself, and I am sure the other people working in this area will offer review time too.

What I would like is a write-up of how this should look like, i.e. a proposal (in a GDoc first) with

• the overall vision and scope
• a list of use-cases we want to support
• a sketch of the tool from the end-user perspective, i.e. command-line flags.

I agree, let's switch over to the other issue.

[pwittrock]

closed in favor of kubernetes/kubernetes#53524