RFC: Kata + Footloose: VM Like containers in Kata

[mcastelino]

Kata + Footloose: VM Like containers in Kata

weaveworks recently open sourced footloose https://github.com/weaveworks/footloose
This consists of tooling around docker and also a set of container images.

This allows containers to look like VM's to a large degree. i.e. systemd is the supervisor inside the container, reaping done by systemd. Ability to install application and allow them to be managed by systemd inside the container. (sshd is setup by default in footloose container images).

We tested footloose with docker with kata set as default runtime and it works.
Also we tested the footlose container images independent of footloose both with docker and kubernetes and that too works.

This opens the door for Kata to actually launch "lightweight VMs". Also given that Kata also supports different Kernel per pod, we can get very close to "ultralight VMs" in Kata.

This is also a very good use case to showcase the custom kernel support we have in Kata.

I propose that we add footloose or footloose container images to our CI to ensure we continue to work.

Footloose container as a Kubernetes POD or a docker container.

So a VM in Kubernetes launched and managed as a container using the Kata runtimeClass.

This is a very interesting use case for people using containers/kubernetes to run CI.
This is very much along the lines of how @awprice et al use docker in Kubernetes using Kata.

/cc @chavafg @ganeshmaharaj @egernst @kata-containers/architecture-committee
Thank you /cc @dlespiau

How to run footloose pod with Kata on Kubernetes

Note: We need to add the pod kernel annotation to this to use it with a custom kernel to make it more complete.

apiVersion: v1
data:
    authorized_keys: |
      ssh-rsa PUT_YOUR_KEY_HERE
kind: ConfigMap
metadata:
  name: ssh-pub-key
---
apiVersion: v1
kind: Pod
metadata:
  name: kataFC29VM
spec:
  runtimeClassName: kata
  volumes:
  - name: runv
    emptyDir:
      medium: "Memory"
  - name: runlockv
    emptyDir:
      medium: "Memory"
  - name: tmpv
    emptyDir:
      medium: "Memory"
  - name: fakecgroup
    hostPath:
      path: /sys/fs/cgroup
  - name: ssh-dir
    emptyDir:
      medium: "Memory"
  - name: ssh-pub-key
    configMap:
      name: ssh-pub-key
      defaultMode: 384
  containers:
  - name: vmcontainer
    image: quay.io/footloose/fedora29:latest
    command: ["/sbin/init"]
    volumeMounts:
    - name: runv
      mountPath: /run
    - name: runlockv
      mountPath: /run/lock
    - name: tmpv
      mountPath: /tmp
    - name: fakecgroup
      readOnly: true
      mountPath: /sys/fs/cgroup
    - name: ssh-dir
      mountPath: /root/.ssh
    - name: ssh-pub-key
      mountPath: /root/.ssh/authorized_keys
      subPath: authorized_keys
  # These containers are run during pod initialization
  initContainers:
  - name: install
    image: busybox
    command: ["sh", "-c", "chmod 700 /root/.ssh"]
    volumeMounts:
    - name: ssh-dir
      mountPath: /root/.ssh

With footloose and with docker

https://gist.github.com/mcastelino/533bd97b3b5690b726bbe82692a28bab

[amshinde]

@mcastelino So are these like machine containers? See this:
#1399 (comment)

[mcastelino]

/cc @GabyCT @chavafg any feedback? Can we add these to the pods we test today.

[GabyCT]

@mcastelino yes, I will add the tests

[GabyCT]

@mcastelino , I have a question about this test, I was not able to connect to the pod ip as it looks it requires a password. What I did is the following:

1. Install footloose
2. Create ssh-key at the host
3. Insert the public key at the configmap.yaml

#
# Copyright (c) 2019 Intel Corporation
#
# SPDX-License-Identifier: Apache-2.0
#
apiVersion: v1
data:
    authorized_keys: |
      ${ssh_key}
kind: ConfigMap
metadata:
  name: ssh-pub-key

4. Create ConfigMap
   $ kubectl create -f "runtimeclass_workloads/footloose-rsa-configmap.yaml"

5. Create pod
   $ kubectl create -f "runtimeclass_workloads/pod-footloose.yaml"

apiVersion: v1
kind: Pod
metadata:
  name: footfed
spec:
  runtimeClassName: kata
  volumes:
  - name: runv
    emptyDir:
      medium: "Memory"
  - name: runlockv
    emptyDir:
      medium: "Memory"
  - name: tmpv
    emptyDir:
      medium: "Memory"
  - name: fakecgroup
    hostPath:
      path: /sys/fs/cgroup
  containers:
  - name: footfed
    image: quay.io/footloose/fedora29:latest
    command: ["/sbin/init"]
    volumeMounts:
    - name: runv
      mountPath: /run
    - name: runlockv
      mountPath: /run/lock
    - name: tmpv
      mountPath: /tmp
    - name: fakecgroup
      readOnly: true
      mountPath: /sys/fs/cgroup

6. Check pod creation
   kubectl wait --for=condition=Ready pod "$pod_name"

7. Get pod ip
   pod_ip=$(kubectl get pod $pod_name --template={{.status.podIP}})

8. Add ip to known_hosts
   sudo ssh-keyscan "$pod_ip" >> ~/.ssh/known_hosts

However when I want to do a
$ ssh root@"$pod_ip" uname -r

It is asking me for a password, is there a way to avoid this

[ganeshmaharaj]

@GabyCT do you have this bit also in youir yaml file for the footloose container?

- name: ssh-pub-key
      mountPath: /root/.ssh/authorized_keys
      subPath: authorized_keys
  # These containers are run during pod initialization
  initContainers:
  - name: install
    image: busybox
    command: ["sh", "-c", "chmod 700 /root/.ssh"]
    volumeMounts:
    - name: ssh-dir
      mountPath: /root/.ssh

The password prompt happens if the ssh server is not able to access the key to authenticate if the person connecting is authorized by viewing the authorized_keys file.

[GabyCT]

@ganeshmaharaj yes, I added to the yaml however I am having the same issue, I created the ssh-key at /tmp/ with the name of mykey and I am still can not connect as is asking me for a password

apiVersion: v1
kind: Pod
metadata:
  name: footfed
spec:
  runtimeClassName: kata
  volumes:
  - name: runv
    emptyDir:
      medium: "Memory"
  - name: runlockv
    emptyDir:
      medium: "Memory"
  - name: tmpv
    emptyDir:
      medium: "Memory"
  - name: fakecgroup
    hostPath:
      path: /sys/fs/cgroup
  - name: ssh-dir
    emptyDir:
      medium: "Memory"
  - name: ssh-pub-key
    configMap:
      name: ssh-pub-key
      defaultMode: 384
  containers:
  - name: vmcontainer
    image: quay.io/footloose/fedora29:latest
    command: ["/sbin/init"]
    volumeMounts:
    - name: runv
      mountPath: /run
    - name: runlockv
      mountPath: /run/lock
    - name: tmpv
      mountPath: /tmp
    - name: fakecgroup
      readOnly: true
      mountPath: /sys/fs/cgroup
    - name: ssh-dir
      mountPath: /root/.ssh
    - name: ssh-pub-key
      mountPath: /root/.ssh/authorized_keys
      subPath: authorized_keys
  # These containers are run during pod initialization
  initContainers:
  - name: install
    image: busybox
    command: ["sh", "-c", "chmod 700 /root/.ssh"]
    volumeMounts:
    - name: ssh-dir
      mountPath: /root/.ssh