Skip to content
Branch: master
Find file History
mythi crypto-perf: use IPC_LOCK to ensure mmap() works
Change SYS_ADMIN to IPC_LOCK capability to ensure DPDK gets to mmap() hugepages.

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Latest commit 08a079e Jun 7, 2019
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
dpdkdrv qat: add kernel mode plugin Apr 25, 2019
kerneldrv qat_plugin: kerneldrv: register VF devices when IOMMU is on May 29, 2019
README.md crypto-perf: use IPC_LOCK to ensure mmap() works Jun 12, 2019
qat_plugin.go qat: add kernel mode plugin Apr 25, 2019

README.md

Build and test Intel® QuickAssist Technology (QAT) device plugin for Kubernetes

Prerequisites

Get source code:

$ mkdir -p $GOPATH/src/github.com/intel/
$ cd $GOPATH/src/github.com/intel/
$ git clone https://github.com/intel/intel-device-plugins-for-kubernetes.git

Build QAT device plugin:

$ cd $GOPATH/src/github.com/intel/intel-device-plugins-for-kubernetes
$ make qat_plugin

Verify kubelet socket exists in /var/lib/kubelet/device-plugins/ directory:

$ ls /var/lib/kubelet/device-plugins/kubelet.sock
/var/lib/kubelet/device-plugins/kubelet.sock

Deploy QAT device plugin directly on the host:

$ sudo $GOPATH/src/github.com/intel/intel-device-plugins-for-kubernetes/cmd/qat_plugin/qat_plugin \
-dpdk-driver igb_uio -kernel-vf-drivers dh895xccvf -max-num-devices 10 -debug
QAT device plugin started
Discovered Devices below:
03:01.0 device: corresponding DPDK device detected is uio0
03:01.1 device: corresponding DPDK device detected is uio1
03:01.2 device: corresponding DPDK device detected is uio2
03:01.3 device: corresponding DPDK device detected is uio3
03:01.4 device: corresponding DPDK device detected is uio4
03:01.5 device: corresponding DPDK device detected is uio5
03:01.6 device: corresponding DPDK device detected is uio6
03:01.7 device: corresponding DPDK device detected is uio7
03:02.0 device: corresponding DPDK device detected is uio8
03:02.1 device: corresponding DPDK device detected is uio9
The number of devices discovered are:10
device-plugin start server at: /var/lib/kubelet/device-plugins/intelQAT.sock
device-plugin registered
ListAndWatch: Sending device response

By default, the device plugin supports these QuickAssist devices: DH895xCC, C62x, C3xxx, and D15xx devices.

Use the kernel-vf-drivers flag to specify the vf Device Driver for the particular QAT device. For more information, refer to Intel QAT Crypto Poll Mode Driver.

-dpdk-driver is set to vfio-pci by default since it is more robust and secure driver compared with igb_uio. See DPDK Linux Driver Guide for more information.

Build QAT device plugin Docker image:

$ cd $GOPATH/src/github.com/intel/intel-device-plugins-for-kubernetes
$ make  intel-qat-plugin

Deploy QAT device plugin as a DaemonSet:

$ cd $GOPATH/src/github.com/intel/intel-device-plugins-for-kubernetes
kubectl create -f deployments/qat_plugin/qat_plugin_default_configmap.yaml
kubectl create -f deployments/qat_plugin/qat_plugin.yaml

Verify QAT device plugin is registered on master:

$ kubectl describe node <node name> | grep qat.intel.com/generic
 qat.intel.com/generic: 10
 qat.intel.com/generic: 10

Consuming QAT device plugin

  1. Build the DPDK image:

    $ cd demo
    $ ./build-image.sh crypto-perf
    

    This command produces a Docker image named crypto-perf.

  2. Deploy a pod to run an example DPDK application named dpdk-test-crypto-perf.

    In the pod specification file, add container resource request and limit. For example, qat.intel.com/generic: <number of devices> for a container requesting QAT devices.

    For a DPDK-based workload, you may need to add hugepage request and limit.

    $ kubectl create -f demo/crypto-perf-dpdk-pod-requesting-qat.yaml
    $ kubectl get pods
      NAME                     READY     STATUS    RESTARTS   AGE
      dpdkqatuio               1/1       Running   0          27m
      intel-qat-plugin-5zgvb   1/1       Running   0          3h
    
    

    Note: If the igb_uio VF driver is used with the QAT device plugin, the workload be deployed with SYS_ADMIN capabilities added.

  3. Manually execute the dpdk-test-crypto-perf application to review the logs:

    $ kubectl exec -it dpdkqatuio bash
    
    $ ./dpdk-test-crypto-perf -l 6-7 -w $QAT1 -- --ptest throughput --\
     devtype crypto_qat --optype cipher-only --cipher-algo aes-cbc --cipher-op \
     encrypt --cipher-key-sz 16 --total-ops 10000000 --burst-sz 32 --buffer-sz 64
    
    
You can’t perform that action at this time.