Hardware transcoding

[colek42]

It looks like the current implementation does not support gpu hardware transcoding. From my understanding, we just need the NVIDIA/AMD drivers in the plex containers to make this happen. I can work on this if you have some direction.

[munnerz]

Yep this won't pass through any additional directories needed for hardware acceleration. Have you got any info on how you'd enable this normally (ideally how to do it in Docker?). We'd then just want some way to set a nodeSelector of some sort for the pod to target the nodes with GPUs.

…

On Tue, 9 Jan 2018 at 20:56, Cole Kennedy ***@***.***> wrote: It looks like the current implementation does not support gpu hardware transcoding. From my understanding, we just need the NVIDIA/AMD drivers in the plex containers to make this happen. I can work on this if you have some direction. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#12>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMbPy6NnY2I2Ls25rS2Z5UQDwqYm-SRks5tI9J0gaJpZM4RYbZu> .

[munnerz]

I should add that you are also able to configure the image that kube-plex uses for the transcoder, so if you were to provide an image of the same version but with the drivers added in, it should work.

There is some more information on using GPUs with Kubernetes here: https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/. Looks like we need to add the following to the container spec;

      resources:
        limits:
          nvidia.com/gpu: 1

This could fold into #6

[colek42]

I'll dive into this issue this weekend.

[tiny-dancer]

@colek42, any luck on this one?

[colek42]

I played with it for about an hour, did not make any progress. Someday I'll hack at it again.

[marcus-grant]

This may be of interest, I won't have time to try it out for at least a week, but basically it goes over over either using nvidia-docker hooks for nVidia devices, and DRI hooks for mesa for intel devices. The latter is probably the most interesting considering many clusters will probably be placed on public clouds running predominantly intel CPUs. There's also some interesting reference links.

ros.org: hardware accelerated containers

[billimek]

I got this working in the plexinc/pms-docker container part of this helm chart, FWIW:

In my case one of the k8s nodes is running as a VM to a guest with a Skylake CPU that is exposing the intel GPU to the ubuntu VM running the k8s node (via iommu & gvt device virtualization). I'm also running the Intel GPU Device Plugin for Kubernetes in order to facilitate exposing the GPU device to pods that request it in a resource limit, e.g.

    resources:
      limits:
       gpu.intel.com/i915: 1

'enabling' this was as easy as adding the above resource setting to the values for this helm chart (after deploying the necessary intel GPU device plugin daemonset)

[munnerz]

I have also got this working with an old nvidia GPU using the nvidia device plugin. It requires:

• Install the NVIDIA device driver (can probably be done using an init container/daemonset and the nvidia install container) - I installed this on my host machine in testing
• Install the NVIDIA kubernetes device plugin
• Add nvidia.com/gpu resource limit to the plex container:

      resources:
        limits:
          nvidia.com/gpu: 1

• Add the NVIDIA_DRIVER_CAPABILITIES=all environment variable to the plex container (this can probably be reduced to just video, but I've not tested)

This works fine even when running plex as a non-root user (as is default in the pmsinc docker images)

I also tried adding 2 nvidia GPUs (as I had a spare lying around). These were visible when running nvidia-smi, however Plex still enforced the only-two-streams-hardware-transcoding limit when I tried to start 4 transcodes at once. I've not had a chance to dig into this further to work out the real reason why, or how to work around this:

root@plex-kube-plex:/# nvidia-smi
Mon Apr 15 14:45:54 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.56       Driver Version: 418.56       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 660     Off  | 00000000:01:00.0 N/A |                  N/A |
| 30%   51C    P0    N/A /  N/A |     97MiB /  1490MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 650 Ti  Off  | 00000000:06:00.0 N/A |                  N/A |
| 24%   36C    P0    N/A /  N/A |     11MiB /   981MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+

[todaywasawesome]

I'm working on getting this to work with the kube-plex on multiple pods but I haven't quite figured out the syntax for adding the limits/requests in the pod creation.
https://github.com/todaywasawesome/kube-plex/blob/trasncode/main.go#L125

My thought was to add them as environment variables on the kube-plex container but I'm stuck having little familiarity with using the client-go to creating the resources this way.

[somerandow]

I'll see if I can take a look at this as well, as this is a feature I would like to make use of if possible.