Skip to content
BranchesTags

Repository files navigation

TrueNAS CSI Driver

A Container Storage Interface (CSI) driver for TrueNAS 25.10.0+, enabling dynamic provisioning of persistent volumes in Kubernetes using TrueNAS storage.

Features

  • NFS volumes - ReadWriteMany (RWX) access mode for shared storage
  • iSCSI volumes - Block storage with ReadWriteOnce (RWO) and ReadWriteMany (RWX) access modes (RWX requires cluster filesystem like GFS2/OCFS2)
  • NVMe-oF/TCP volumes - Block storage over NVMe over Fabrics (TCP) with optional DH-CHAP authentication
  • Dynamic provisioning - Automatic volume creation and deletion
  • Volume expansion - Online resize of volumes
  • Snapshots and clones - CSI snapshot support for backup and cloning
  • CHAP authentication - Secure iSCSI connections
  • ZFS compression - LZ4, ZSTD, GZIP, and other algorithms
  • ZFS encryption - Dataset-level encryption with key management
  • Automatic snapshot scheduling - Periodic snapshots via StorageClass
  • TrueNAS Websocket API - Uses the modern TrueNAS Websocket API

Requirements

TrueNAS

  • TrueNAS SCALE 25.10.0+
  • API access enabled
  • At least one ZFS pool configured

Kubernetes

Node Requirements

  • NFS volumes: No additional requirements
  • iSCSI volumes: open-iscsi package installed on worker nodes
  • NVMe-oF volumes: nvme_tcp/nvme_fabrics kernel modules available on worker nodes (the node DaemonSet loads them); requires TrueNAS SCALE 25.10+ with the NVMe-oF target service enabled

Quick Start

  1. Create an API key in TrueNAS

    • Log into TrueNAS web UI
    • Navigate to your profile → API Keys
    • Create a new API key and copy it

  2. Configure the driver

    # Edit the deployment manifest
vi deploy/truenas-csi-driver.yaml

    Update the ConfigMap with your TrueNAS connection details and the Secret with your API key.

  3. Deploy the driver

    kubectl apply -f deploy/truenas-csi-driver.yaml

  4. Create a StorageClass and PVC

    kubectl apply -f examples/storageclass-nfs.yaml
kubectl apply -f examples/pvc-nfs.yaml

Installation

Prerequisites

Install the snapshot controller (required for snapshot support):

kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/deploy/kubernetes/snapshot-controller/rbac-snapshot-controller.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml

Deploy the Driver

  1. Edit deploy/truenas-csi-driver.yaml with your configuration
  2. Apply the manifest: 
    kubectl apply -f deploy/truenas-csi-driver.yaml

Verify Installation

# Check driver pods are running
kubectl get pods -n truenas-csi

# Verify CSI driver is registered
kubectl get csidrivers

Non-Standard Kubelet Paths

The default deployment manifest uses /var/lib/kubelet as the kubelet root directory. Some Kubernetes distributions use a different path. If your distribution uses a non-standard path, you must update the following in deploy/truenas-csi-driver.yaml before deploying:

  1. All hostPath values containing /var/lib/kubelet
  2. The DRIVER_REG_SOCK_PATH environment variable
  3. The --kubelet-registration-path argument
  4. The mountPath for the kubelet-dir volume mount on the csi-node container
Distribution Kubelet Path
Standard Kubernetes /var/lib/kubelet (default)
MicroK8s /var/snap/microk8s/common/var/lib/kubelet
K3s /var/lib/rancher/k3s/agent/kubelet

Important: The kubelet-dir mountPath must match the hostPath. If they differ, NFS mounts will succeed inside the CSI container but will not propagate to kubelet, causing pods to see local storage instead of NFS.

MicroK8s Mount Propagation

MicroK8s runs inside a snap with its own mount namespace. For CSI mount propagation to work, the host root filesystem must have shared propagation before MicroK8s starts:

sudo mount --make-rshared /
microk8s start

To make this persistent across reboots, create a systemd unit:

sudo tee /etc/systemd/system/microk8s-mount-propagation.service <<EOF
[Unit]
Description=Ensure shared mount propagation for MicroK8s
Before=snap.microk8s.daemon-containerd.service

[Service]
Type=oneshot
ExecStart=/bin/mount --make-rshared /
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl enable microk8s-mount-propagation

Configuration

Driver Configuration (ConfigMap)

Setting Description Example
truenasURL WebSocket URL to TrueNAS API wss://10.0.0.100/api/current
truenasInsecure Skip TLS verification true (for self-signed certs)
defaultPool Default ZFS pool for volumes tank
nfsServer NFS server address 10.0.0.100
iscsiPortal iSCSI portal address 10.0.0.100:3260
nvmeofPortal NVMe-oF portal address (optional; auto-derived) 10.0.0.100:4420
iscsiIQNBase Base IQN for iSCSI targets iqn.2024-01.com.example

StorageClass Parameters

General Parameters

Parameter Description Values
protocol Storage protocol nfs, iscsi, nvmeof
pool ZFS pool (overrides default) pool name
datasetPath Parent path for volume datasets, relative to the pool (no pool prefix, no leading/trailing /, no ..). If unset, volumes are created at the pool root (pool/<pvc-name>); e.g. k8s/iscsi → pool/k8s/iscsi/<pvc-name> relative path
compression ZFS compression algorithm OFF, LZ4, GZIP[-1|-9], ZSTD[-1..-9], ZLE, LZJB
sync ZFS sync mode STANDARD, ALWAYS, DISABLED
sparse Thin-provision the ZVOL (iSCSI/NVMe-oF); default false true, false

Delete-time behavior (optional): forceDelete (true/false) forces removal of busy resources; deleteExtentsWithTarget (true/false, default true) removes the iSCSI extent along with its target.

NFS Parameters

Parameter Description Example
nfs.hosts Allowed hosts 10.0.0.0/8,192.168.1.0/24
nfs.networks Allowed networks 10.0.0.0/8
nfs.mountOptions Client mount options hard,nfsvers=4.1
nfs.mapAllUser NFS user mapping (default: root) postgres
nfs.mapAllGroup NFS group mapping (default: wheel) postgres
nfs.rootSquash Squash all access to the mapped user (default: true). Set false for no_root_squash so a pod fsGroup can chown the volume root — required for ownership-sensitive non-root workloads (e.g. PostgreSQL/CNPG) false

By default an NFS share squashes all client access to a single user (mapall, root:wheel). Ownership-sensitive workloads that run as a non-root user (such as PostgreSQL/CloudNativePG) need to own their data directory, which mapall cannot provide. Set nfs.rootSquash: "false" to switch the share to no_root_squash: incoming root is preserved so the kubelet (via the driver's fsGroupPolicy: File) can chown the volume root to the pod's fsGroup, and non-root UIDs are no longer squashed. Requires the workload to set a pod securityContext.fsGroup. See examples/storageclass-nfs-fsgroup.yaml.

iSCSI Parameters

Parameter Description Values
volblocksize ZVOL block size 512, 1K, 2K, 4K, 8K, 16K, 32K, 64K, 128K
iscsi.blocksize iSCSI logical block size 512, 1024, 2048, 4096
iscsi.iqn-base Override the IQN base (auto-derived from the appliance's iscsi.global.basename by default) IQN string
iscsi.initiators Allowed initiator IQNs comma-separated
iscsi.chapUser CHAP username string
iscsi.chapSecret CHAP password (12-16 chars) string
iscsi.chapPeerUser Mutual CHAP peer user string
iscsi.chapPeerSecret Mutual CHAP peer password string
iscsi.multipathEnabled Enable multipath for the session (node-side); default false true, false
iscsi.persistentSessions Keep the iSCSI session persistent (node-side); default false true, false

IPv4 only: iSCSI portals must be IPv4. The pinned csi-lib-iscsi mis-parses IPv6 portal addresses, so iSCSI staging fails on IPv6-only clusters — use NFS there. The driver fails fast with a clear error if an IPv6 iSCSI portal is configured.

NVMe-oF Parameters

NVMe-oF also uses the volblocksize parameter above. DH-CHAP authentication is optional.

Parameter Description Values
nvmeof.hostNQN Authorized host NQN (required for DH-CHAP) nqn.2014-08.org.nvmexpress:uuid:...
nvmeof.dhchapKey DH-CHAP host key DHHC-1:00:...
nvmeof.dhchapCtrlKey Mutual DH-CHAP controller key DHHC-1:00:...
nvmeof.dhchapHash DH-CHAP hash (default SHA-256) SHA-256, SHA-384, SHA-512
nvmeof.dhchapDHGroup DH group 2048-BIT, 3072-BIT, 4096-BIT, 6144-BIT, 8192-BIT

Snapshot Task Parameters

Parameter Description Values
snapshot.schedule Cron schedule (5 fields) 0 0 * * *
snapshot.retention Retention period 1-365
snapshot.retentionUnit Retention unit HOUR, DAY, WEEK, MONTH, YEAR
snapshot.naming Naming schema auto-%Y-%m-%d_%H-%M
snapshot.recursive Include child datasets true, false

Encryption Parameters

Parameter Description Values
encryption Enable encryption true, false
encryption.algorithm Encryption algorithm AES-256-GCM, AES-128-CCM
encryption.passphrase Passphrase (min 8 chars) string
encryption.key Hex-encoded key (64 chars) string
encryption.generateKey Auto-generate key true, false

Examples

See the examples/ folder for sample configurations:

  • storageclass-nfs.yaml - Basic NFS StorageClass
  • storageclass-nfs-compressed.yaml - NFS with ZSTD compression
  • storageclass-nfs-fsgroup.yaml - NFS for ownership-sensitive non-root workloads (no_root_squash + pod fsGroup)
  • storageclass-iscsi.yaml - Basic iSCSI StorageClass
  • storageclass-iscsi-chap.yaml - iSCSI with CHAP authentication
  • storageclass-nvmeof.yaml - Basic NVMe-oF/TCP StorageClass
  • storageclass-nvmeof-dhchap.yaml - NVMe-oF with DH-CHAP authentication
  • storageclass-encrypted.yaml - Encrypted storage
  • pvc-nfs.yaml / pvc-iscsi.yaml / pvc-nvmeof.yaml - PVC examples
  • pod-with-pvc.yaml - Pod using a PVC
  • volumesnapshotclass.yaml / volumesnapshot.yaml - Snapshot examples

Building

Build the binary

make build

Build container images

# Build Alpine-based image (standard Kubernetes)
make docker-build

# Build UBI-based image (Red Hat OpenShift certification)
make build-ubi

Push to quay.io

# Login to quay.io
docker login quay.io

# Push UBI image to quay.io/truenas_solutions
make push-ubi

# Push all images (driver, operator, bundle)
make push-all

Run tests

make test

Container Images

Image Description
ghcr.io/truenas/truenas-csi CSI driver (Alpine-based, for standard Kubernetes)
quay.io/truenas_solutions/truenas-csi CSI driver (UBI-based, for Red Hat OpenShift)
quay.io/truenas_solutions/truenas-csi-operator Kubernetes operator
quay.io/truenas_solutions/truenas-csi-operator-bundle OLM bundle for OperatorHub

Running the Demo

For an interactive demonstration of all driver features using a local Kind cluster, see docs/demo.md.

OpenShift

The TrueNAS CSI Driver supports Red Hat OpenShift 4.20+ and is designed for OperatorHub distribution.

Quick Start (OpenShift)

  1. Install via OperatorHub

    • Navigate to Operators > OperatorHub
    • Search for "TrueNAS CSI"
    • Click Install

  2. Create credentials secret

    apiVersion: v1
kind: Secret
metadata:
  name: truenas-api-credentials
  namespace: truenas-csi
stringData:
  api-key: "YOUR-API-KEY"

  3. Create TrueNASCSI resource

    apiVersion: csi.truenas.io/v1alpha1
kind: TrueNASCSI
metadata:
  name: truenas
spec:
  truenasURL: "wss://your-truenas-ip/api/current"
  credentialsSecret: "truenas-api-credentials"
  defaultPool: "tank"
  nfsServer: "your-truenas-ip"

OpenShift Documentation

Demo Scripts

Interactive demo scripts are provided to test the CSI driver:

Standard Kubernetes (Kind)

# Set TrueNAS connection details in deploy/truenas-csi-driver.yaml, then:
./demo-simple.sh

OpenShift (CRC/OpenShift Local)

# Set environment variables
export TRUENAS_IP=192.168.1.100
export TRUENAS_API_KEY=your-api-key
export TRUENAS_POOL=tank

# Run the demo
./demo-openshift.sh

Both demos provide interactive menus to test NFS/iSCSI provisioning, volume expansion, snapshots, and cloning.

Contributing

License

GNU General Public License 3.0

About

TrueNAS Official K8's & Openshift CSI Drivers

Resources

Readme
GPL-3.0 license
Activity
Custom properties

Stars

91 stars

Watchers

5 watching

Forks

22 forks
Report repository

Releases

Packages

Contributors

Languages