Design discussion: sharing NVDEC/NVENC in multi‑tenant DeepStream pipelines .

[maishivamhoo123]

Context

Issue #2002 outlines the problem: DeepStream pipelines contend for fixed‑function NVDEC/NVENC engines, causing failures when multiple video pods share a GPU. HAMi already virtualises compute and memory, but not the video hardware.

Objective

This discussion is to explore how HAMi could safely share or isolate NVDEC/NVENC across containers. I’d like to gather feedback from the community—especially anyone running DeepStream in Kubernetes—and agree on a feasible first step.

Potential approaches (open for debate)

1. User‑space proxy/scheduler – A sidecar or HAMi‑managed service that serialises access to the decoder/encoder, enforcing a “one user at a time” policy with proper fencing.
2. Capacity isolation – Expose a resource like nvidia.com/decoder-streams with a max‑streams limit per container, enforced either in the NVIDIA driver (if future MPS/cgroup support arrives) or via a shim layer.
3. Leveraging existing NVIDIA hooks – Could nvidia-container-toolkit hooks or environment variables (e.g., NVDEC_NUM_DECODERS) be used to limit visibility? What are the gaps?
4. Kernel/driver changes – Advocate for or prototype NVIDIA MPS extensions for video engines (longer term).

Questions for the community

• Are you hitting the NVDEC/NVENC contention problem in production? What workarounds are you using today?
• Would a simple fencing mechanism (only one container accesses the engine at a time) be an acceptable MVP, or is fine‑grained stream limiting essential from day one?
• Which approach would fit most cleanly into HAMi’s existing virtualisation architecture?
• Any prior art (other projects, NVIDIA roadmap) we should be aware of?

/cc @archlitchi @Shouren

[Shouren]

@maishivamhoo123 AFAK, the dynamic mig can help isolating the NVENC/NVDEC