Kubernetes connection within porter operator

[JeremyGHutchins]

I have a porter bundle that has a kubeconfig parameter that maps to /root/.kube/config in the porter bundle. It works great from the cli, but now I'm trying to use it to install our bundle via the operator instead. I was assuming that connections to the current K8s cluster would be handled, since it is already running within the cluster, however when I run it I get...

DEBUG name:    kubernetes
DEBUG pkgDir: /cnab/app/mixins/kubernetes
DEBUG file:     
DEBUG stdin:
install:
- kubernetes:
    description: 'Installing: Workbench Operator Namespaces'
    manifests:
    - manifests/namespace
    - manifests/default
    - manifests/wb
    name: kubernetes-kinetica-k8s-operator-namespaces
    wait: true

/cnab/app/cnab/app/mixins/kubernetes/runtimes/kubernetes-runtime install --debug
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Error: error running command kubectl apply -f manifests/namespace --wait: exit status 1
err: error running command kubectl apply -f manifests/namespace --wait: exit status 1
Error: mixin execution failed: exit status 1

How do I properly setup the connection to kubernetes in the bundle installer? Do I need to provide a kubeconfig as a parameter and how would I do that?

[carolynvs]

I spoke with @vdice today about a long term solution: Specifying a credential source for the kubeconfig so that it can be injected with the in-cluster config when a kubeconfig is not specified. I will work on getting that added.

It would look something like this:

parameters;
  - name: kubeconfig
     type: file
     source:
         in-cluster: true # use the in-cluster configuration when a kubeconfig is not provided

In the meantime, I have tested out using a base64 encoded string containing the kubeconfig as a parameter (credentials can only be specified as a credential set and I don't think you are using those). So you can do something like this for now:

apiVersion: porter.sh/v1
kind: Installation
metadata:
  name: mysql
spec:
  reference: "carolynvs/mysql:v0.1.3"
  action: "install"
  parameters:
    namespace: mysql
    mysql-name: carolyn-mysql
    kubeconfig: YXBpVmVyc2lvbjogdjEKY2x1c.... # generated with cat kube.config > base64 -w 0

[JeremyGHutchins]

Permission wise, I'm wondering if it might be a good plan to create service accounts for each install (that could then in turn be added to the individual pods), so that permissions could be granted in a targeted manner - then you could provide the name of the service account to run under. Otherwise, porter would have to have global rights to everything, which doesn't tend to make security teams happy...

[carolynvs]

You can configure the service account under which the bundle runs like this:

apiVersion: porter.sh/v1
kind: Installation
metadata:
  name: mysql
spec:
  reference: "carolynvs/mysql:v0.1.3"
  action: "upgrade"
  agentConfig:
    installationServiceAccount: mysql-acct

You can also customize the service account that porter runs with (because porter needs more permissions than the invocation image does, for example to create jobs).

apiVersion: porter.sh/v1
kind: Installation
metadata:
  name: mysql
spec:
  reference: "carolynvs/mysql:v0.1.3"
  action: "upgrade"
  agentConfig:
    serviceAccount: porter-acct

The agent config is a custom resource that can either be embedded in an installation, defined in the same namespace as the installation or in the operator's namespace. The configuration is inherited so that you could define just once how you want Porter's agent to run and all installations will use that by default.

[carolynvs]

Docs is definitely on the todo list as we move from a POC to v1. #30 😁

[JeremyGHutchins]

Fantastic! Thank you! Option 1 worked as advertized - moved the credential to a parameter...

parameters: 
  - name: kubeconfig
    type: file
    source:
      in-cluster: true
    path: /root/.kube/config

Will try to look at the service account later, as I think that will be a good longer-term solution.

[lucidprogrammer]

parameters;
  - name: kubeconfig
     type: file
     source:
         in-cluster: true

@carolynvs is this supported now?

[carolynvs]

There's no need to check on issues that are still open. If it's open, no one has implemented it yet. 😀