Load plugins for apis server. (#2805)

* Add unimplemented kapp-controller plugin.

* Load plugins

* Update readme

* Update README with info about pluggable apis

* UPdate readme with file locations.

cmd/kubeapps-apis/.gitignore

@@ -0,0 +1 @@
+devel/*

cmd/kubeapps-apis/Makefile

@@ -12,6 +12,12 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+# Build the helm and kapp package plugins with the output in the devel directory.
+build-plugins:
+	go build -o devel/kapp-controller-packages-v1alpha1-plugin.so -buildmode=plugin plugins/kapp_controller/packages/v1alpha1/main.go
+	go build -o devel/helm-operator-packages-v1alpha1-plugin.so -buildmode=plugin plugins/helm_operator/packages/v1alpha1/main.go
+
+
 # Ensure the required version of cli tooling in tools/tools.go is installed.
 # The versions are stored in ../go.mod as usual and this same approach is used in
 # both octant and pinniped for build tooling.
@@ -33,10 +39,10 @@ cli-dependencies:
 generate:
 	buf generate
 
-run:
-	go run main.go
+run: build-plugins
+	go run main.go --plugin-dir devel/
 
 test:
 	go test ./...
 
-.PHONY: cli-dependencies generate run test
+.PHONY: build-plugins cli-dependencies generate run test

cmd/kubeapps-apis/README.md

@@ -16,6 +16,17 @@ Together, this enables the best of both worlds: a well known and used Interface
 
 Finally, we've also chosen to use the [buf](https://buf.build/) tool for generating the code from the proto files. In the past we've used `protoc` (proto buffer compiler) and its extensions directly, but `buf` allows you to specify a simple yaml config instead, and also provides a `lint` command to ensure that your choice of API structure follows best practise, as well as ensuring you're aware when you break backwards compatability.
 
+## Plug-able
+
+The kubeapps-apis service uses the standard [go plugin package](https://golang.org/pkg/plugin/) to be able to load api plugins at run-time.
+
+Each plugin consists of 2 source files (and some generated files):
+
+* A `.proto` file defining the service that uses the messages defined in relevant part of kubeappsapis.core, in `./proto/kubeappsapis/plugins/<plugin-name>`,
+* A `main.go` that compiles to an .so file for that plugin. This `main.go` has two public functions: one to register the plugin with a GRPC server and one to register the plugin for the http handler as well as the implementation for the server. This may be split into further modules as the complexity of the plugin grows. This file is under `./plugins/<plugin-name>`
+
+With this structure, the kubeapps-apis' main.go simply loads the `.so` files from the specified plugin dirs and register them when starting. You can see this in the [kubeapps-apis/server/server.go](server/server.go) file.
+
 ## CLI
 
 Similar to most go commands, we've used [Cobra](https://github.com/spf13/cobra) for the CLI interface. Currently there is only a root command to run server, but we may later add a `version` subcommand or a `new-plugin` subcommand, but even without these it provides a lot of useful defaults for config, env var support etc.
@@ -27,12 +38,13 @@ You can run `make run` to run the currently stubbed service.
 ```bash
 make run
 
+I0514 14:14:52.969498 1932386 server.go:129] Successfully registered plugin "/home/michael/dev/vmware/kubeapps/cmd/kubeapps-apis/devel/helm-operator-packages-v1alpha1-plugin.so"
+I0514 14:14:52.975884 1932386 server.go:129] Successfully registered plugin "/home/michael/dev/vmware/kubeapps/cmd/kubeapps-apis/devel/kapp-controller-packages-v1alpha1-plugin.so"
 I0511 11:39:56.444553 4116647 server.go:25] Starting server on :50051
 ```
 
 You can then verify the (currently stubbed) configured plugins endpoint via http:
 
-
 ```bash
 curl http://localhost:50051/core/plugins/v1alpha1/configured-plugins
 
@@ -50,6 +62,15 @@ grpcurl -plaintext localhost:50051 kubeappsapis.core.plugins.v1alpha1.PluginsSer
 }
 ```
 
+The packages endpoints for the helm-operator and kapp-controller plugins are unimplemented:
+
+```bash
+curl http://localhost:50051/kapp_controller/packages/v1alpha1
+{"code":12, "message":"method GetAvailablePackages not implemented", "details":[]}
+
+curl http://localhost:50051/helm/packages/v1alpha1
+{"code":12, "message":"method GetAvailablePackages not implemented", "details":[]}
+```
 
 ## Hacking