The csi-attacher is part of Kubernetes implementation of Container Storage Interface (CSI).
In short, it's an external controller that monitors
VolumeAttachment objects and attaches/detaches volumes to/from nodes. Full design can be found at Kubernetes proposal at container-storage-interface.md
There is no plan to implement a generic external attacher library, csi-attacher is the only external attacher that exists. If this proves false in future, splitting a generic external-attacher library should be possible with some effort.
External attacher follows controller pattern and uses informers to watch for
PersistentVolume create/update/delete events. It filters out
VolumeAttachment instances with
Attacher==<CSI driver name> and processes these events in workqueues with exponential backoff. Real handling is deferred to
Handler interface has two implementations, trivial and real one.
Trivial handler will be used for CSI drivers that don't support
ControllerPublish calls and marks all
VolumeAttachment as attached. It does not use any finalizers. This attacher can also be used for testing.
"Real" attacher talks to CSI over socket (
/run/csi/socket by default, configurable by
-csi-address). The attacher tries to connect for
-connection-timeout (1 minute by default), allowing CSI driver to start and create its server socket a bit later.
The attacher then:
Discovers the supported attacher name by
GetPluginInfocalls. The attacher only processes
VolumeAttachmentinstances that have
ControllerGetCapabilitiesto find out if CSI driver supports
ControllerPublishcalls. It degrades to trivial mode if not.
VolumeAttachmentinstances and attaches/detaches volumes:
VolumeAttachmentthat wants to attach PV with
- A finalizer is added to
VolumeAttachmentinstance to preserve the object after deletion so we can detach the volume.
- A finalizer is added to referenced PV instance to preserve the PV. Attacher needs information from the PV to detach the volume.
AttachmentMetadatais saved to
- On any error, the
VolumeAttachmentis re-queued with exponential backoff.
- A finalizer is removed from
VolumeAttachment. At this point, the API server is going to delete this instance and "deleted
VolumeAttachment" event will be received.
- Pokes PV queue with name of the detached PV. This triggers removal of finalizer on PV, if needed.
Processes added/updated PV to remove finalizer on PVs:
- Ignore PVs that don't have DeletionTimestamp.
- Checks that the PV is not used by any
- Removes Attacher's finalizer on the PV if so.
- On any error, the PV is re-queued with exponential backoff.
Both PV queue and
VolumeAttachment queue run in parallel. To ensure that removal of PV finalizers work without races:
- The controller attaches PVs only when the PV has no DeletionTimestamp and has attacher's finalizer.
- The controller removes finalizer only from PVs that have DeletionTimestamp.
As consequence, the attacher must be available until all PVs that refer to the CSI driver are removed. Even fully detached PVs have attacher's finalizer that is removed only after the PV is marked for deletion.
Secondary cache and locks was considered to keep a map PV -> list of VolumeAttachments that use the PV. Attacher's finalizer could be removed from a PV immediately after the last VolumeAttachment was deleted. Keeping this map is either racy or requires long critical sections with complicated error recovery.
Dummy attacher watches for
VolumeAttachment instances with
Attacher=="csi/dummy" and marks them attached. It does not use any finalizers and is useful for testing.
To run dummy attacher in
$ csi-attacher -dummy -kubeconfig ~/.kube/config -v 5
Running on command line
For debugging, it's possible to run the attacher on command line:
$ csi-attacher -kubeconfig ~/.kube/config -v 5 -csi-address /run/csi/socket
Running in a deployment
It is necessary to create a new service account and give it enough privileges to run the attacher. We provide one omnipotent yaml file that creates everything that's necessary, however it should be split into multiple files in production.
$ kubectl create deploy/kubernetes/deployment.yaml
Note that the attacher does not scale with more replicas. Only one attacher is elected as leader and running. The others are waiting for the leader to die. They re-elect a new active leader in ~15 seconds after death of the old leader.
We use dep for management of
vendor/k8s.io is manually copied from
staging/ directory of work-in-progress API for CSI, namely https://github.com/kubernetes/kubernetes/pull/54463.
Community, discussion, contribution, and support
Learn how to engage with the Kubernetes community on the community page.
You can reach the maintainers of this project at:
Code of conduct
Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.