Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nvidia-container-cli fails to enable Vulkan in containers #55

Open
foresto opened this issue May 16, 2019 · 6 comments

Comments

@foresto
Copy link

commented May 16, 2019

I'm using lxc with the mount hook that calls nvidia-container-cli, and it works for running nvidia-smi and opengl applications in the container. However, vulkan applications, including vulkaninfo, vkcube, and games, all fail. For example:

$ vulkaninfo
Cannot create Vulkan instance.
/build/vulkan-tools-_xrZWD/vulkan-tools-1.1.101.0+dfsg1/vulkaninfo/vulkaninfo.c:884: failed with VK_ERROR_INCOMPATIBLE_DRIVER

Digging a little deeper with strace, I found that /usr/share/vulkan/icd.d/nvidia_icd.json is missing in the container. When I manually exported it to the container's filesystem, I got a new error:

$ vulkaninfo
The NVIDIA driver was unable to open 'libnvidia-glvkspirv.so.430.14'. This library is required at run time.

WARNING: [Loader Message] Code 0 : terminator_CreateInstance: Failed to CreateInstance in ICD 0. Skipping ICD.
Cannot create Vulkan instance.
/build/vulkan-tools-_xrZWD/vulkan-tools-1.1.101.0+dfsg1/vulkaninfo/vulkaninfo.c:884: failed with VK_ERROR_INCOMPATIBLE_DRIVER

Following the trail of errors, I tried manually exporting /usr/lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.430.14 into the container. At that point, vulkan applications started working.

So, it looks like nvidia-container-cli and/or its libraries are failing to expose at least these files to the container:

/usr/share/vulkan/icd.d/nvidia_icd.json
/usr/lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.*
/usr/lib/i386-linux-gnu/libnvidia-glvkspirv.so.*

Can you folks please fix this?

@RenaudWasTaken

This comment has been minimized.

Copy link
Collaborator

commented May 16, 2019

Hello!

Vulkan isn't supported in containers yet. One of the reasons is we haven't discussed with someone familiar with Vulkan.

Ideally I would be interested in having some confirmation that this is the only thing we need to do to enable Vulkan in nvidia containers.

@foresto

This comment has been minimized.

Copy link
Author

commented May 16, 2019

Oh? That's surprising, since Vulkan is listed under nvidia-container-runtime's supported driver capabilities. Would it not make sense to expose those files now even if they're officially unsupported, since they're part of the same driver package whose other files are already exposed, and since I can confirm that their presence makes things work?

In any case, thanks for the info. I suppose I'll have to continue mounting those three files manually until support is added here.

@stgraber

This comment has been minimized.

Copy link
Contributor

commented Jun 24, 2019

It'd be great if the common files could at least get added as that's causing issues for our users who use LXD containers to run steam and other games.
(As can be seen in the linked LXD issue)

@RenaudWasTaken

This comment has been minimized.

Copy link
Collaborator

commented Jun 27, 2019

@stgraber we've started working on this and are hoping to enable it in the next few weeks :)
cc @guptaNswati

@guptaNswati

This comment has been minimized.

Copy link
Contributor

commented Jun 27, 2019

@stgraber thats the goal. Right now I am trying to find all the needed deps across different versions, so that we can make it generic. One of the libs it needs is libnvidia_GLX.so that libnvidia-container already exposes if the graphics option is used but not others. And icd_loader comes from nvidia_utils package that also needs to be added.

@stgraber

This comment has been minimized.

Copy link
Contributor

commented Jun 28, 2019

Excellent, thanks for working on this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.