Clear Container with GVTg Setup Guide
This page contains info on setup GVT-g for clear container, so container VM can use GPU device. Containers are immensely popular in the cloud world. With Clear Containers we're working on a way to improve security of containers by using Intel® Virtualization Technology (link is external) (Intel® VT).
We set out to build Clear Containers by leveraging the isolation of virtual-machine technology along with the deployment benefits of containers. As part of this, we let go of the "generic PC hardware" notion traditionally associated with virtual machines; we're not going to pretend to be a standard PC that is compatible with just about any OS on the planet. Why? By shedding the legacy baggage, our model provides a dramatically improved experience: it’s slimmer, simpler, safer, and a lot speedier!
Make sure to use latest 4.11 RC or final kernel which have GVT-g support.
GVT-g kernel config can reference https://github.com/01org/gvt-linux/wiki/GVTg_Setup_Guide
https://github.com/clearcontainers/qemu/tree/qemu-lite-v2.9.0
./configure --disable-tools --disable-libssh2 --disable-tcmalloc
--disable-glusterfs --disable-seccomp --disable-{bzip2,snappy,lzo}
--disable-usb-redir --disable-libusb --disable-libnfs
--disable-tcg-interpreter --disable-debug-tcg --disable-libiscsi
--disable-rbd --disable-spice --disable-attr --disable-cap-ng
--disable-linux-aio --disable-brlapi --disable-vnc-{jpeg,png,sasl}
--disable-rdma --disable-bluez --disable-fdt --disable-curl
--disable-curses --disable-sdl --disable-gtk --disable-tpm
--disable-vte --disable-vnc --disable-xen --disable-opengl
--disable-slirp --enable-trace-backend=nop
--target-list=x86_64-softmmu
make clean
make -j10
Build Kernel Source:
git clone https://github.com/01org/gvt-linux.git
cd gvt-linux
git checkout gvt-staging
echo ""|make oldconfig
Then make sure to enable CONFIG_DRM_I915_GVT and CONFIG_DRM_I915_GVT_KVMGT in ".config", which depends on CONFIG_VFIO_MDEV and CONFIG_VFIO_MDEV_DEVICE.
make -j8 && make modules_install
You can download clear linux image from Download Clear Linux Image. Newer version should also work.
xz -d clear-(the version you choose)-kvm.img.xz
After boot with GVT-g enabled host kernel, you can create a clear container with the script as follows:
#!/bin/sh
QEMU_BIN=~/qemu/x86_64-softmmu/qemu-system-x86_64
KERNEL=~/gvt-linux/vmlinux
KERNEL_CMD="reboot=k panic=1 rw tsc=reliable no_timer_check noreplace-smp root=PARTUUID=4aed2554-435f-4918-9f01-29022631a92c clocksource=kvm-clock consol e=ttyS0 loglevel=8 pci=lastbus=0"
#create mdev
UUID=1bdc0b0a-258a-11e7-9f96-a711621dfeb2
echo "$UUID" > /sys/devices/pci0000:00/0000:00:02.0/mdev_supported_types/i915-GVTg_V4_2/create
echo $UUID created
set -x
$QEMU_BIN -machine q35,accel=kvm,kernel_irqchip,smm=off,nosmbus,nopit,nofw \
-cpu host -m 1024,maxmem=20G,slots=2 -smp 6 \
-kernel $KERNEL -append "$KERNEL_CMD" \
-drive format=qcow2,if=virtio,file=/root/img/clear-8800-kvm.qcow2 \
-nodefaults \
-nographic \
-serial stdio -monitor pty \
-device virtio-net-pci,netdev=net0,mac=DE:AD:BE:EF:07:94 -netdev tap,id=net0 \
-device vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/$UUID,addr=04.0
echo "Remove $UUID"
echo 1 > /sys/devices/pci0000:00/0000:00:02.0/$UUID/remove
Note1: this is just an example script, you need to adjust for your setting.
Note2: make sure to use the correct root option(root=/dev/vda1 or root=PARTUUID=(the uuid in the clear image))
you can do as follows to get the PARTUUID:
sudo fdisk -lu clear-(the version you choose)-kvm.img
sudo mount -o loop,offset=1048576 clear-(the version you choose)-kvm.img /mnt/
cd /mnt/loader/entries
cat Clear-Linux-kvm-(the OS version).conf
First verify if drm/i915 module is loaded in clear linux guest.
cd /sys/module
ls
Then try to copy several intel-gpu-tools cases and install required libraries e.g libdrm, libpciaccess, etc.