Demo aggregate core apis. (#2904)

* Register plugins that implement core services for aggregation.

* Add implementation for core.packages.v1alpha1.GetAvailablePackages

* Example of aggregate core api in README

* Remove obsolete comment and switch to Infof.

* Note the possibility of using go-routines for processing each plugin in the aggregate API

* Add link to upstream issue report.

cmd/kubeapps-apis/README.md

@@ -27,6 +27,10 @@ Each plugin consists of 2 source files (and some generated files):
 
 With this structure, the kubeapps-apis' main.go simply loads the `.so` files from the specified plugin dirs and register them when starting. You can see this in the [kubeapps-apis/server/server.go](server/server.go) file.
 
+## Aggregated
+
+When plugins are registered, they are also checked to see if they implement a core API (currently the only one is core.packages.v1alpha1). If they do, they are registered for use by the corresponding core API for aggregating results across plugins. See below for an example.
+
 ## CLI
 
 Similar to most go commands, we've used [Cobra](https://github.com/spf13/cobra) for the CLI interface. Currently there is only a root command to run server, but we may later add a `version` subcommand or a `new-plugin` subcommand, but even without these it provides a lot of useful defaults for config, env var support etc.
@@ -128,6 +132,33 @@ $ curl -s http://localhost:8080/plugins/kapp_controller/packages/v1alpha1/packag
 }
 ```
 
+Or you can query the core API to get an aggregation of all package repositories (or packages) across the relevant plugins. This output will include the additional plugin field for each item:
+
+```bash
+curl -s http://localhost:8080/core/packages/v1alpha1/packagerepositories | jq .
+{
+  "repositories": [
+    {
+      "name": "bitnami",
+      "namespace": "flux-system",
+      "url": "https://charts.bitnami.com/bitnami",
+      "plugin": {
+        "name": "fluxv2.packages",
+        "version": "v1alpha1"
+      }
+    },
+    {
+      "name": "demo-package-repository",
+      "url": "k8slt/corp-com-pkg-repo:1.0.0",
+      "plugin": {
+        "name": "kapp_controller.packages",
+        "version": "v1alpha1"
+      }
+    }
+  ]
+}
+```
+
 Of course, you will need to have the appropriate Flux HelmRepository or Carvel PackageRepository available in your cluster.
 ## Hacking
 

cmd/kubeapps-apis/buf.gen.yaml

@@ -8,6 +8,22 @@ plugins:
     out: gen
     opt:
       - module=github.com/kubeapps/kubeapps/cmd/kubeapps-apis/gen
+      # The following option controls whether the generated interfaces require
+      # the `mustEmbedUnimplemented**Server` method. This method is useful to
+      # ensure that when you add a method to a proto service and regenerate
+      # files, they will automatically implement the new method and return
+      # "Unimplemented", as opposed to not compiling until the new method is
+      # added explicitly. But a plugin can still embed the
+      # `Unimplemented**Server` struct, it's just not enforced. The
+      # unintentional downside of enforcing it via the
+      # `mustEmbedUnimplemented**Server` for us is that the
+      # `mustEmbedUnimplemented**Server` method is not exposed, which means that
+      # *only* implementations in the same package (ie. with access to the same
+      # unexposed method) can satisfy the interface. We explicitly want plugins
+      # defined in other packages to be able to satisfy a generic gRPC server
+      # interface.
+      # Created https://github.com/grpc/grpc-go/issues/4500 to discuss upstream.
+      - require_unimplemented_servers=false
   - name: grpc-gateway
     out: gen
     opt:

cmd/kubeapps-apis/docs/kubeapps-apis.swagger.json

@@ -14,6 +14,9 @@
     {
       "name": "PluginsService"
     },
+    {
+      "name": "PackagesService"
+    },
     {
       "name": "FluxV2PackagesService"
     },
@@ -32,6 +35,82 @@
     "application/json"
   ],
   "paths": {
+    "/core/packages/v1alpha1/packagerepositories": {
+      "get": {
+        "operationId": "PackagesService_GetPackageRepositories",
+        "responses": {
+          "200": {
+            "description": "A successful response.",
+            "schema": {
+              "$ref": "#/definitions/v1alpha1GetPackageRepositoriesResponse"
+            }
+          },
+          "default": {
+            "description": "An unexpected error response.",
+            "schema": {
+              "$ref": "#/definitions/rpcStatus"
+            }
+          }
+        },
+        "parameters": [
+          {
+            "name": "cluster",
+            "description": "A cluster name can be provided if multiple clusters are configured,\notherwise the current cluster will be assumed.",
+            "in": "query",
+            "required": false,
+            "type": "string"
+          },
+          {
+            "name": "namespace",
+            "description": "A namespace can be provided if the package repositories for a specific namespace\nare requested, when supported by the packaging format.",
+            "in": "query",
+            "required": false,
+            "type": "string"
+          }
+        ],
+        "tags": [
+          "PackagesService"
+        ]
+      }
+    },
+    "/core/packages/v1alpha1/packages": {
+      "get": {
+        "operationId": "PackagesService_GetAvailablePackages",
+        "responses": {
+          "200": {
+            "description": "A successful response.",
+            "schema": {
+              "$ref": "#/definitions/v1alpha1GetAvailablePackagesResponse"
+            }
+          },
+          "default": {
+            "description": "An unexpected error response.",
+            "schema": {
+              "$ref": "#/definitions/rpcStatus"
+            }
+          }
+        },
+        "parameters": [
+          {
+            "name": "cluster",
+            "description": "A cluster name can be provided if multiple clusters are configured,\notherwise the current cluster will be assumed.",
+            "in": "query",
+            "required": false,
+            "type": "string"
+          },
+          {
+            "name": "namespace",
+            "description": "A namespace can be provided if the packages available for install in a\nspecific namespace are required.",
+            "in": "query",
+            "required": false,
+            "type": "string"
+          }
+        ],
+        "tags": [
+          "PackagesService"
+        ]
+      }
+    },
     "/core/plugins/v1alpha1/configured-plugins": {
       "get": {
         "summary": "GetConfiguredPlugins returns a map of short and longnames for the configured plugins.",