connection plugin kubectl for kubernetes.

[xuxinkun]

SUMMARY

This PR is the connection plugin for kubernetes. It can deploy the playbook directly into containers of kubernetes pod using the local kubectl.

kubernetes has a tool to enter container called kubectl. It has similar feature as docker exec.
So, by making use of kubectl, we can alse achieve the goal treating the kubernetes pod as container of docker.

After this PR, inventory example could be like this.

[k8s-pod]
mypod ansible_connection=kubernetes ansible_kubernetes_extra_args="-s http://101.88.64.106:8080" ansible_kubernetes_namespace=xuxinkun ansible_kubernetes_container=master

The following parameters are processed by this connector:

• ansible_kubernetes_container: If the pod has more than one container, the container name must be specified.
• ansible_kubernetes_namespace: The namespace of pod. If it is not set, it will be default.
• ansible_kubernetes_extra_args: Could be a string with any additional arguments understood by kubectl, which are not command specific. This parameter is mainly used to configure the options of kubectl.

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

• contrib/inventory/kubernetes.py
• lib/ansible/plugins/connection/kubernetes.py

ANSIBLE VERSION

devel

ADDITIONAL INFORMATION

I also see that there is a similar PR #24960. But I do not agree with it. First, kubernetes version has changed almost every two months. So, using python library kubernetes can not be compatible with each version. And it did not implement fetch_file.

I take use of kubectl instead of python library kubernetes.

[root@ab22a1cf08be ~]# kubectl help exec
Execute a command in a container.

Examples:
  # Get output from running 'date' from pod 123456-7890, using the first container by default
  kubectl exec 123456-7890 date
  
  # Get output from running 'date' in ruby-container from pod 123456-7890
  kubectl exec 123456-7890 -c ruby-container date
  
  # Switch to raw terminal mode, sends stdin to 'bash' in ruby-container from pod 123456-7890
  # and sends stdout/stderr from 'bash' back to the client
  kubectl exec 123456-7890 -c ruby-container -i -t -- bash -il

Options:
  -c, --container='': Container name. If omitted, the first container in the pod will be chosen
  -p, --pod='': Pod name
  -i, --stdin=false: Pass stdin to the container
  -t, --tty=false: Stdin is a TTY

Usage:
  kubectl exec POD [-c CONTAINER] -- COMMAND [args...] [options]

And this has not been changed and will not be changed for a long period of time.

But the options has been changed a lot. So, I set ansible_kubernetes_extra_args to replace it. Ansible user can set it easily.

[xuxinkun]

@bcoca Pls review and check. Kubernetes is more and more popular. I hope this could be merged to help kubernetes users.

[ansibot]

The test ansible-test sanity --test pep8 failed with the following error:

lib/ansible/plugins/connection/kubernetes.py:77:29: E231 missing whitespace after ','

click here for bot help

[Ledger45]

Replacing python api with kubectl is a good idea

[chrrrles]

Did a cursory connection test using the raw module and several Openshift containers. Worked good for that.

[chrrrles]

Is this based on JCPowermac's connection plugin?

[xuxinkun]

based on docker connection. I missed it. I will fix it right now.

[m8cool]

Wonderful feature. I cannot wait to use this plugin.

[chouseknecht]

@xuxinkun @chrrrles

Did some testing with OpenShift as well. I could only get it to work with the raw module. Why is that?

To get it to work I created a soft link for kubectl -> oc, as I don't have kubectl installed. Then I created the following inventory file:

django-3-6jt7z ansible_kubernetes_namespace=demo

In the above django-3-6jt7z is the name of the pod. It only contains a single command. And, demo is the name of the project.

Again, the question is, why does the raw module work, but command does not? And a second question, why does gather_facts not work?

In both cases (i.e. using command, or enabling gather_facts), I get the following error:

UNREACHABLE! => {"changed": false, "msg": "Authentication or permission failure. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote temp path in ansible.cfg to a path rooted in \"/tmp\". Failed command was: ( umask 77 && mkdir -p \"` echo /.ansible/tmp/ansible-tmp-1499887864.83-262240610026274 `\" && echo ansible-tmp-1499887864.83-262240610026274=\"` echo /.ansible/tmp/ansible-tmp-1499887864.83-262240610026274 `\" ), exited with result 1", "unreachable": true

FWIW, here's my playbook using raw:

- name: Talk to containers on OpenShift
  hosts: django-3-6jt7z
  gather_facts: no
  connection: kubernetes
  tasks:

    - raw: date
      register: output

    - debug: var=output.stdout

UPDATE:

I think the issue is likely that the container is running as an unknown user. That's the default OpenShift behavior. It runs your container as a random user. So perhaps the only thing that will ever work in a default OpenShift installation is the raw command.

$ oc exec django-3-6jt7z whoami
whoami: cannot find name for user ID 1000060000

[chrrrles]

UPDATE:
Chris - you are correct about the user id -- The "pet" pod below is deployed with a root service account, so is able to run modules.

Hi Chris,

I am testing on my local minishift, using a kubectl installed by brew. Try running the playbook on containers with a mounted, writable volume ... I believe that is required in order for this connection to work with modules other than raw. Here is a playbook and a run on a "pet" pod with writable volumes.

- hosts: mypod
  tasks:
          - name: echo "Hello"
            raw: echo "HELLO raw"
            register: output
          - debug: var=output.stdout
          - name: command echo "Hello"
            command: echo "HELLO command"
            register: output
          - debug: var=output.stdout
          - name: command echo "Hello"
            shell: echo "HELLO shell"
            register: output
          - debug: var=output.stdout

PLAY [mypod] *****************************************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] *******************************************************************************************************************************************************************************************************************************************
ok: [aerospike-2-4md2v]

TASK [echo "Hello"] **********************************************************************************************************************************************************************************************************************************************
changed: [aerospike-2-4md2v]

TASK [debug] *****************************************************************************************************************************************************************************************************************************************************
ok: [aerospike-2-4md2v] => {
    "failed": false,
    "output.stdout": "HELLO raw\n"
}

TASK [command echo "Hello"] **************************************************************************************************************************************************************************************************************************************
changed: [aerospike-2-4md2v]

TASK [debug] *****************************************************************************************************************************************************************************************************************************************************
ok: [aerospike-2-4md2v] => {
    "failed": false,
    "output.stdout": "HELLO command"
}

TASK [command echo "Hello"] **************************************************************************************************************************************************************************************************************************************
changed: [aerospike-2-4md2v]

TASK [debug] *****************************************************************************************************************************************************************************************************************************************************
ok: [aerospike-2-4md2v] => {
    "failed": false,
    "output.stdout": "HELLO shell"
}

PLAY RECAP *******************************************************************************************************************************************************************************************************************************************************
aerospike-2-4md2v          : ok=7    changed=3    unreachable=0    failed=0

[xuxinkun]

@chouseknecht I don't know about how oc works with exec. But this pulgin work well with origin kubectl.
Here is my inventory:

[k8s]
centos-pod ansible_kubernetes_extra_args="-s http://10.8.64.106:8080" ansible_kubernetes_namespace=xuxinkun

and my playbook:

- name: Talk to containers on OpenShift
  hosts: k8s
  connection: kubernetes
  tasks:
    - raw: date
      register: output
    - debug: var=output.stdout
    - command: date
      register: cmd_output
    - debug: var=cmd_output.stdout

And here is my result:

[root@ab22a1cf08be ~]# ansible-playbook playbook  -v
Using /etc/ansible/ansible.cfg as config file

PLAY [Talk to containers on OpenShift] ******************************************************************************

TASK [Gathering Facts] **********************************************************************************************
ok: [centos-pod]

TASK [raw] **********************************************************************************************************
changed: [centos-pod] => {"changed": true, "failed": false, "rc": 0, "stderr": "", "stdout": "Thu Jul 13 01:52:35 UTC

TASK [debug] ********************************************************************************************************
ok: [centos-pod] => {
    "failed": false, 
    "output.stdout": "Thu Jul 13 01:52:35 UTC 2017\n"
}

TASK [command] ******************************************************************************************************
changed: [centos-pod] => {"changed": true, "cmd": ["date"], "delta": "0:00:01.081603", "end": "2017-07-13 01:52:37.81_lines": [], "stdout": "Thu Jul 13 01:52:36 UTC 2017", "stdout_lines": ["Thu Jul 13 01:52:36 UTC 2017"]}

TASK [debug] ********************************************************************************************************
ok: [centos-pod] => {
    "cmd_output.stdout": "Thu Jul 13 01:52:36 UTC 2017", 
    "failed": false
}

PLAY RECAP **********************************************************************************************************
centos-pod                 : ok=5    changed=2    unreachable=0    failed=0 

[xuxinkun]

@chouseknecht BTW, as I know, the kubectl exec could not specify the remote user right now. But docker client has achieved that. Maybe kubectl will achieve it in the future. I'm not sure. So my connection plugin can not support specifying remote user. Users can only use the user which container started with.
And also, some images or containers does not have python or dd. And this plugin can not work with them too.

[jcpowermac]

@chouseknecht take a look at my test project for my iteration for the kube connection plugin. I used an example from my team's repo that deals with arbitrary uid. Arbitrary uid just needs to be handled in the Docker image. I would figure this implementation would have no problem with any modules as long as the uid issue was handled.

[chouseknecht]

@xuxinkun

Will work on pushing these changes into your branch today.

As far as discussion around parameter specifics, totally open to a discussion. If anyone has ideas, or wants to make changes, now's the time. I've simply attempted to surface the Python client parameters, keeping the names and meaning consistent with the client. But again, open to modifying this approach, and iterating on it to make it better.

[jstoja]

Maybe add the namespace as a connection parameter?

[chouseknecht]

@jstoja

Good idea. Namespace is definitely included.

Added a bunch of variables, and support for env variables too. I think parameters are now pretty well aligned with the modules.

[xuxinkun]

How about using kubectl_verify_ssl instead of verify_ssl?

[chouseknecht]

Yep. That's a bug. Fixing.

[xuxinkun]

I prefer to use server instead of host. Because host in ansible points to the target of operation.
BTW, kubectl also use -s, --server string The address and port of the Kubernetes API server

[chouseknecht]

I know. There's an alias for 'server'. Calling it host keeps it consistent with the modules and Python client. As a user, you can use server or host. The Python client calls it host, and uses HOST in the env variable.

[xuxinkun]

How about adding a SERVER env variable?

[chouseknecht]

Done.

[xuxinkun]

Should we also provide a env option for kubectl_container?

[chouseknecht]

Doesn't seem like it would be necessary, as container names aren't really known ahead of time. I'm envisioning pod and container names coming from an inventory script.

[xuxinkun]

If pod has only one container, container name can be ignored. But if pod has more than one container, container name is necessary to be specified.

[chouseknecht]

Added.

[chouseknecht]

Added an oc connection plugin that subclasses kubectl. That should suffice, if you only have access to the oc binary, or just want to test it directly for some reason. It does have slightly different options. Plus, it has an rsync command, which may prove useful.

[xuxinkun]

Remove these blank lines.