Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker container #113

Closed
xspython opened this issue Dec 11, 2019 · 8 comments
Closed

docker container #113

xspython opened this issue Dec 11, 2019 · 8 comments

Comments

@xspython
Copy link

I experimented with the docker nvida M60 video card, and the result was not very good

The environment is as follows:

1.docker nvida cuda

2.turbovnc-2.2.3

3.VirtualGL-2.5.2-1

4.xfce4

Running vglrun glxsphere S64 FPS is much smaller than running glxsphere S64 directly

Does virtualgl not support docker

@ffeldhaus
Copy link

Can you have a look at
https://github.com/ffeldhaus/docker-xpra-html5-opengl
and check if it solves your issues? It uses xpra instead of turbovnc, but you should be able to use a similar setup with turbovnc by checking the Dockerfile (or just use xpra).

@dcommander
Copy link
Member

dcommander commented Sep 15, 2020

Here's how I made it work on CentOS 7, using a host-side TurboVNC session and 3D X server:

  • As root, uninstall the O/S-supplied version of Docker and install the docker-ce package (instructions here.)

  • As root, install the nvidia-docker2 package (instructions here.)

  • Example Dockerfile for running GLXspheres:

    FROM nvidia/opengl:1.0-glvnd-runtime-centos7
    
    ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES},graphics,compat32
    
    RUN yum-config-manager --add-repo https://virtualgl.org/pmwiki/uploads/Downloads/VirtualGL.repo \
     && yum -y install \
        yum-utils \
        xorg-x11-utils \
        xorg-x11-xauth \
        glx-utils \
        VirtualGL.x86_64 VirtualGL.i386 \
     && yum clean all \
     && rm -rf /etc/ld.so.cache \
     && rm -rf /var/cache/ldconfig/* \
     && rm -rf /tmp/*
    
  • In a TurboVNC session, run the Docker image (in my case, built from the Dockerfile above) using

    xauth merge /etc/opt/VirtualGL/vgl_xauth_key
    rm ~/.Xauthority.docker
    xauth nlist $DISPLAY :0.0 | sed -e 's/^..../ffff/' | xauth -f ~/.Xauthority.docker nmerge -
    docker run -it \
      --gpus 1 \
      --rm \
      -e DISPLAY \
      -v ~/.Xauthority.docker:/root/.Xauthority \
      -v /tmp/.X11-unix:/tmp/.X11-unix \
      {docker-image} \
      bash
    

    where {docker-image} is the ID or tag of your image.

    NOTES:

    • Leave out the first line if you are not limiting 3D X server access to the vglusers group.
    • Replace :0.0 with the display name of your 3D X server, if necessary.
  • In the Docker instance, you should now be able to execute vglrun /opt/VirtualGL/bin/glxspheres64 or vglrun /opt/VirtualGL/bin/glxspheres.

Note that the MIT-SHM extension will not work, because it relies on a host-side shared memory segment, and Docker containers use a separate IPC namespace. However, that limitation doesn't seem to have a visible impact on performance.

@dcommander
Copy link
Member

dcommander commented Sep 15, 2020

With the EGL back end, it's a bit easier:

  • As root, uninstall the O/S-supplied version of Docker and install the docker-ce package (instructions here.)
  • As root, install the nvidia-docker2 package (instructions here.)
  • Example Dockerfile for running GLXspheres:
    FROM nvidia/opengl:1.0-glvnd-runtime-centos7
    
    ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES},graphics,compat32
    
    RUN yum-config-manager --add-repo https://virtualgl.org/pmwiki/uploads/Downloads/VirtualGL.repo \
     && yum -y install \
        yum-utils \
        xorg-x11-utils \
        xorg-x11-xauth \
        glx-utils \
        https://s3.amazonaws.com/virtualgl-pr/dev/linux/VirtualGL-2.6.80.x86_64.rpm \
        https://s3.amazonaws.com/virtualgl-pr/dev/linux/VirtualGL-2.6.80.i386.rpm \
     && yum clean all \
     && rm -rf /etc/ld.so.cache \
     && rm -rf /var/cache/ldconfig/* \
     && rm -rf /tmp/*
    
  • In a TurboVNC session, run the Docker image (in my case, built from the Dockerfile above) using
    rm ~/.Xauthority.docker
    xauth nlist $DISPLAY | sed -e 's/^..../ffff/' | xauth -f ~/.Xauthority.docker nmerge -
    docker run -it \
      --gpus 1 \
      --device=/dev/dri \
      --rm \
      -e DISPLAY \
      -v ~/.Xauthority.docker:/root/.Xauthority \
      -v /tmp/.X11-unix:/tmp/.X11-unix \
      {docker-image} \
      bash
    
    where {docker-image} is the ID or tag of your image.
  • In the Docker instance, you should now be able to execute vglrun -d /dev/dri/card0 /opt/VirtualGL/bin/glxspheres64 or vglrun -d /dev/dri/card0 /opt/VirtualGL/bin/glxspheres.

@dcommander
Copy link
Member

For completeness, here's how to accomplish the same thing using a TurboVNC session running inside the Docker container. There are probably about 1000 different ways to do this, as well as many ways to improve upon this methodology, but this and the previous comments should at least be a reasonable starting point. Please ask follow-up questions if you have trouble adapting one of these recipes.

GLX and EGL back ends

  • As root, uninstall the O/S-supplied version of Docker and install the docker-ce package (instructions here.)
  • As root, install the nvidia-docker2 package (instructions here.)

GLX back end (3D X server)

  • Example Dockerfile for running GLXspheres and TurboVNC:

    FROM nvidia/opengl:1.0-glvnd-runtime-centos7
    
    ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES},graphics,compat32
    
    RUN yum-config-manager --add-repo https://virtualgl.org/pmwiki/uploads/Downloads/VirtualGL.repo \
     && yum-config-manager --add-repo https://turbovnc.org/pmwiki/uploads/Downloads/TurboVNC.repo \
     && yum -y install epel-release.noarch \
     && yum -y install \
        yum-utils \
        xorg-x11-utils \
        xorg-x11-xauth \
        glx-utils \
        VirtualGL.x86_64 VirtualGL.i386 \
        turbovnc \
     && yum -y groupinstall "MATE Desktop" \
     && yum clean all \
     && rm -rf /etc/ld.so.cache \
     && rm -rf /var/cache/ldconfig/* \
     && rm -rf /tmp/*
    
  • In an SSH session on the host, run the Docker image (in my case, built from the Dockerfile above) using

    xauth merge /etc/opt/VirtualGL/vgl_xauth_key
    rm ~/.Xauthority.docker
    xauth nlist :0.0 | sed -e 's/^..../ffff/' | xauth -f ~/.Xauthority.docker nmerge -
    docker run -it \
      --gpus 1 \
      --rm \
      -p 15900-15999:5900-5999 \
      -v ~/.Xauthority.docker:/etc/opt/VirtualGL/vgl_xauth_key \
      -v /tmp/.X11-unix/X0:/tmp/.X11-unix/X0 \
      {docker-image} \
      /opt/TurboVNC/bin/vncserver -fg -wm mate-session
    

    where {docker-image} is the ID or tag of your image.

    NOTES:

    • Leave out the first line if you are not limiting 3D X server access to the vglusers group.
    • Replace :0.0 with the display name of your 3D X server, if necessary.
  • Connect to the TurboVNC session on {host}::{port}, where {port} is 15900 + the TurboVNC display number.

  • In the TurboVNC session, you should now be able to execute vglrun /opt/VirtualGL/bin/glxspheres64 or vglrun /opt/VirtualGL/bin/glxspheres.

EGL back end

  • Example Dockerfile for running GLXspheres and TurboVNC:

    FROM nvidia/opengl:1.0-glvnd-runtime-centos7
    
    ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES},graphics,compat32
    
    RUN yum-config-manager --add-repo https://virtualgl.org/pmwiki/uploads/Downloads/VirtualGL.repo \
     && yum-config-manager --add-repo https://turbovnc.org/pmwiki/uploads/Downloads/TurboVNC.repo \
     && yum -y install epel-release.noarch \
     && yum -y install \
        yum-utils \
        xorg-x11-utils \
        xorg-x11-xauth \
        glx-utils \
        https://s3.amazonaws.com/virtualgl-pr/dev/linux/VirtualGL-2.6.80.x86_64.rpm \
        https://s3.amazonaws.com/virtualgl-pr/dev/linux/VirtualGL-2.6.80.i386.rpm \
        turbovnc \
     && yum -y groupinstall "MATE Desktop" \
     && yum clean all \
     && rm -rf /etc/ld.so.cache \
     && rm -rf /var/cache/ldconfig/* \
     && rm -rf /tmp/*
    
  • In an SSH session on the host, run the Docker image (in my case, built from the Dockerfile above) using

    docker run -it \
      --gpus 1 \
      --device=/dev/dri \
      --rm \
      -p 15900-15999:5900-5999 \
      {docker-image} \
      /opt/TurboVNC/bin/vncserver -fg -wm mate-session
    

    where {docker-image} is the ID or tag of your image.

  • Connect to the TurboVNC session on {host}::{port}, where {port} is 15900 + the TurboVNC display number.

  • In the TurboVNC session, you should now be able to execute vglrun -d /dev/dri/card0 /opt/VirtualGL/bin/glxspheres64 or vglrun -d /dev/dri/card0 /opt/VirtualGL/bin/glxspheres.

@dcommander
Copy link
Member

NOTE: The instructions above for running TurboVNC in the Docker container don't work quite right for multiple sessions or multiple users. Rather than specifying -p 15900-15999:5900-5999, it seems better to specify --expose 5901 -P, which causes Docker to map VNC display :1 in the container to an available ephemeral port on the host. You can then use docker ps to figure out which port was mapped.

@dcommander
Copy link
Member

https://github.com/dcommander/virtualgl_docker_examples now contains my latest Docker/VirtualGL/TurboVNC examples.

@ehfd
Copy link

ehfd commented Nov 13, 2020

https://github.com/ehfd/docker-nvidia-egl-desktop

MATE Desktop container for NVIDIA GPUs without using an X Server, directly accessing the GPU with EGL to emulate GLX using VirtualGL and TurboVNC. Does not require /tmp/X11-unix host sockets.

@dcommander
Copy link
Member

@ehfd Please stop posting duplicate comments. You are creating noise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants