Switch branches/tags
Nothing to show
Find file History

README.md

Introduction

The Kubernetes iSCSI implementation can connect to iSCSI devices via open-iscsi and multipathd on Linux. Currently supported features are

  • Connecting to one portal
  • Mounting a device directly or via multipathd
  • Formatting and partitioning any new device connected
  • CHAP authentication

Prerequisites

This example expects there to be a working iSCSI target to connect to. If there isn't one in place then it is possible to setup a software version on Linux by following these guides

Creating the pod with iSCSI persistent storage

Once you have configured the iSCSI initiator, you can create a pod based on the example iscsi.yaml. In the pod YAML, you need to provide targetPortal (the iSCSI target's IP address and port if not the default port 3260), target's iqn, lun, and the type of the filesystem that has been created on the lun, and readOnly boolean. No initiator information is required. If you have more than one target portals for a single IQN, you can mention other portal IPs in portals field.

If you want to use an iSCSI offload card or other open-iscsi transports besides tcp, setup an iSCSI interface and provide iscsiInterface in the pod YAML. The default name for an iscsi iface (open-iscsi parameter iface.iscsi_ifacename) is in the format transport_name.hwaddress when generated by iscsiadm. See open-iscsi or openstack for detailed configuration information.

Note: If you have followed the instructions in the links above you may have partitioned the device, the iSCSI volume plugin does not currently support partitions so format the device as one partition or leave the device raw and Kubernetes will partition and format it one first mount.

CHAP Authentication

To enable one-way or two-way CHAP authentication for discovery or session, following these steps.

  • Set chapAuthDiscovery to true for discovery authentication.
  • Set chapAuthSession to true for session authentication.
  • Create a CHAP secret and set secretRef to reference the CHAP secret.

Example can be found at iscsi-chap.yaml

CHAP Secret

As illustrated in chap-secret.yaml, the secret must have type kubernetes.io/iscsi-chap and consists of the following keys:

---
apiVersion: v1
kind: Secret
metadata:
  name: chap-secret
type: "kubernetes.io/iscsi-chap"  
data:
  discovery.sendtargets.auth.username: 
  discovery.sendtargets.auth.password: 
  discovery.sendtargets.auth.username_in: 
  discovery.sendtargets.auth.password_in: 
  node.session.auth.username: 
  node.session.auth.password: 
  node.session.auth.username_in: 
  node.session.auth.password_in: 

These keys map to those used by Open-iSCSI initiator. Detailed documents on these keys can be found at Open-iSCSI

Create CHAP secret before creating iSCSI volumes and Pods

# kubectl create -f examples/volumes/iscsi/chap-iscsi.yaml

Once the pod config is created, run it on the Kubernetes master:

kubectl create -f ./your_new_pod.yaml

Here is the example pod created and expected output:

# kubectl create -f examples/volumes/iscsi/iscsi.yaml
# kubectl get pods
NAME      READY     STATUS    RESTARTS   AGE
iscsipd   2/2       RUNNING   0           2m

On the Kubernetes node, verify the mount output

For a non mpio device the output should look like the following

# mount |grep kub
/dev/sdb on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.15:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
/dev/sdb on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw type ext4 (ro,relatime,data=ordered)
/dev/sdc on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.16:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
/dev/sdc on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw type ext4 (rw,relatime,data=ordered)
/dev/sdd on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.17:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
/dev/sdd on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw type ext4 (rw,relatime,data=ordered)

And for a node with mpio enabled the expected output would be similar to the following

# mount |grep kub
/dev/mapper/mpatha on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.15:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
/dev/mapper/mpatha on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-ro type ext4 (ro,relatime,data=ordered)
/dev/mapper/mpathb on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.16:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
/dev/mapper/mpathb on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw type ext4 (rw,relatime,data=ordered)
/dev/mapper/mpathc on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.17:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
/dev/mapper/mpathb on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw type ext4 (rw,relatime,data=ordered)

If you ssh to that machine, you can run docker ps to see the actual pod.

# docker ps
CONTAINER ID        IMAGE                                  COMMAND             CREATED             STATUS              PORTS               NAMES
3b8a772515d2        kubernetes/pause                       "/pause"            6 minutes ago       Up 6 minutes                            k8s_iscsipd-rw.ed58ec4e_iscsipd_default_f527ca5b-6d87-11e5-aa7e-080027ff6387_d25592c5

Run docker inspect and verify the container mounted the host directory into the their /mnt/iscsipd directory.

# docker inspect --format '{{ range .Mounts }}{{ if eq .Destination "/mnt/iscsipd" }}{{ .Source }}{{ end }}{{ end }}' f855336407f4
/var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-ro

# docker inspect --format '{{ range .Mounts }}{{ if eq .Destination "/mnt/iscsipd" }}{{ .Source }}{{ end }}{{ end }}' 3b8a772515d2
/var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw

Analytics