This repository contains a framework for developing plugins for the Kubernetes device plugins framework, along with a number of device plugin implementations utilizing that framework.
Table of Contents
- Device Plugins Operator
- XeLink XPU-Manager sidecar
- Workload Authors
- Pre-built plugin images
- Helm charts
Prerequisites for building and running these device plugins include:
- Appropriate hardware and drivers
- A fully configured [Kubernetes cluster]
- A working [Go environment], of at least version v1.16.
The below sections detail existing plugins developed using the framework.
GPU Device Plugin
The GPU device plugin provides access to discrete and integrated Intel GPU device files.
The demo subdirectory contains both a GPU plugin demo video
and an OpenCL sample deployment (
FPGA Device Plugin
The FPGA device plugin supports FPGA passthrough for the following hardware:
- Intel® Arria® 10 devices
- Intel® Stratix® 10 devices
The FPGA plugin comes as three parts.
Refer to each individual sub-components documentation for more details. Brief overviews of the sub-components are below.
The FPGA device plugin is responsible for
discovering and reporting FPGA devices to
The FPGA admission controller webhook is responsible for performing mapping from user-friendly function IDs to the Interface ID and Bitstream ID that are required for FPGA programming. It also implements access control by namespacing FPGA configuration information.
CRI-O Prestart Hook
The FPGA prestart CRI-O hook performs discovery of the requested FPGA function bitstream and programs FPGA devices based on the environment variables in the workload description.
QAT Device Plugin
SGX Device Plugin
The SGX device plugin allows workloads to use Intel® Software Guard Extensions (Intel® SGX) on platforms with SGX Flexible Launch Control enabled, e.g.,:
- 3rd Generation Intel® Xeon® Scalable processor family, code-named “Ice Lake”
- Intel® Xeon® E3 processor
- Intel® NUC Kit NUC7CJYH
The Intel SGX plugin comes in three parts.
Brief overviews of the Intel SGX sub-components are given below.
The SGX device plugin is responsible for discovering
and reporting Intel SGX device nodes to
Containers requesting Intel SGX resources in the cluster should not use the device plugins resources directly.
Intel SGX Admission Webhook
The Intel SGX admission webhook is responsible for performing Pod mutations based on
sgx.intel.com/quote-provider pod annotation set by the user. The purpose
of the webhook is to hide the details of setting the necessary device resources
and volume mounts for using Intel SGX remote attestation in the cluster. Furthermore,
the Intel SGX admission webhook is responsible for writing a pod/sandbox
sgx.intel.com/epc annotation that is used by Kata Containers to dynamically
adjust its virtualized Intel SGX encrypted page cache (EPC) bank(s) size.
Intel SGX EPC memory registration
The Intel SGX EPC memory available on each node is registered as a Kubernetes extended resource using node-feature-discovery (NFD). An NFD Node Feature Rule is installed as part of SGX device plugin operator deployment and NFD is configured to register the Intel SGX EPC memory extended resource.
Containers requesting Intel SGX EPC resources in the cluster use
sgx.intel.com/epc resource which is of
DSA Device Plugin
The DSA device plugin supports acceleration using the Intel Data Streaming accelerator(DSA).
DLB Device Plugin
The DLB device plugin supports Intel Dynamic Load Balancer accelerator(DLB).
IAA Device Plugin
The IAA device plugin supports acceleration using the Intel Analytics accelerator(IAA).
Device Plugins Operator
To simplify the deployment of the device plugins, a unified device plugins operator is implemented.
Currently the operator has support for the DSA, DLB, FPGA, GPU, IAA, QAT, and Intel SGX device plugins. Each device plugin has its own custom resource definition (CRD) and the corresponding controller that watches CRUD operations to those custom resources.
XeLink XPU-Manager Sidecar
To support interconnected GPUs in Kubernetes, XeLink sidecar is needed.
The XeLink XPU-Manager sidecar README gives information how the sidecar functions and how to use it.
The demo subdirectory contains a number of demonstrations for a variety of the available plugins.
For workloads to get accesss to devices managed by the plugins, the
Pod spec must specify the hardware resources needed:
spec: containers: - name: demo-container image: <registry>/<image>:<version> resources: limits: <device namespace>/<resource>: X
The summary of resources available via plugins in this repository is given in the list below.
Device Namespace : Registered Resource(s)
wq-user-[shared or dedicated]
fpga.intel.com: custom, see mappings
wq-user-[shared or dedicated]
For information on how to develop a new plugin using the framework or work on development task in this repository, see the Developers Guide.
Supported Kubernetes Versions
Releases are made under the github releases area. Supported releases and matching Kubernetes versions are listed below:
|release-0.28||Kubernetes 1.28 branch v1.28.x||supported|
|release-0.27||Kubernetes 1.27 branch v1.27.x||supported|
|release-0.26||Kubernetes 1.26 branch v1.26.x||supported|
|release-0.25||Kubernetes 1.25 branch v1.25.x||unsupported|
|release-0.24||Kubernetes 1.24 branch v1.24.x||unsupported|
|release-0.23||Kubernetes 1.23 branch v1.23.x||unsupported|
|release-0.22||Kubernetes 1.22 branch v1.22.x||unsupported|
|release-0.21||Kubernetes 1.21 branch v1.21.x||unsupported|
|release-0.20||Kubernetes 1.20 branch v1.20.x||unsupported|
|release-0.19||Kubernetes 1.19 branch v1.19.x||unsupported|
|release-0.18||Kubernetes 1.18 branch v1.18.x||unsupported|
|release-0.17||Kubernetes 1.17 branch v1.17.x||unsupported|
|release-0.15||Kubernetes 1.15 branch v1.15.x||unsupported|
|release-0.11||Kubernetes 1.11 branch v1.11.x||unsupported|
Note: Device plugins leverage the Kubernetes v1 API. The API itself is GA (generally available) and does not change between Kubernetes versions. One does not necessarily need to use the latest Kubernetes cluster with the latest device plugin version. Using a newer device plugins release should work without issues on an older Kubernetes cluster. One possible exception to this are the device plugins CRDs that can vary between versions.
Project's release cadence is tied to Kubernetes release cadence. Device plugins release typically follows a couple of weeks after the Kubernetes release. There can be some delays on the releases due to required changes in the pull request pipeline. Once the content is available in the
main branch and CI & e2e validation PASSes, release branch will be created (e.g. release-0.26). The HEAD of release branch will also be tagged with the corresponding tag (e.g. v0.26.0).
Patch releases (e.g. 0.26.3) are done on a need basis if there are security issues or minor fixes requested for specific version. Fixes are always cherry-picked from the
main branch to the release branches.
Pre-built plugin images
Pre-built images of the plugins are available on the Docker hub. These images are automatically built and uploaded to the hub from the latest main branch of this repository.
Release tagged images of the components are also available on the Docker hub, tagged with their release version numbers in the format x.y.z, corresponding to the branches and releases in this repository.
Note: the default deployment files and operators are configured with
IfNotPresent and can be changed with
All of the source code required to build intel-device-plugins-for-kubernetes
is available under Open Source licenses. The source code files identify external Go
modules used. Binaries are distributed as container images on
DockerHub*. Those images contain license texts and source code under
Device Plugins Helm Charts are located in Intel Helm Charts repository Intel Helm Charts. This is another way of distributing Kubernetes resources of the device plugins framework.
To add repo:
helm repo add intel https://intel.github.io/helm-charts