Document how to use VTK/vtki in Docker containers & Travis

[daniellivingston]

Trying to run any VTKI rendering within a Dockerized Jupyter notebook causes a kernel crash, with the message:

ERROR: In ../Rendering/OpenGL2/vtkXOpenGLRenderWindow.cxx, line 452
vtkXOpenGLRenderWindow (0x560a53bb7ce0): bad X server connection. DISPLAY=[I 00:12:28.476 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 0839b810-c699-472b-bd57-11c5cfed7fa5 restarted

I believe this is a more fundamental issue than VTKI (I have a VTK issue open here), but I am hoping that someone else using VTKI has experienced this and has a workaround, or that there is a way to run VTKI in 'headless' (virtual framebuffer) mode.

[banesullivan]

You’ll have to set up the headless display and some other dependencies. First check out my answer to this question:

You’ll have to install libgl1-mesa-dev and xvfb to your docker then run a start script similar to the following to initialize the display when the session starts:

#!/bin/bash
export DISPLAY=:99.0
which Xvfb
Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 &
sleep 3
exec "$@"

Then head over to this repository and try to copy some of my set up scripts there.

Before long, I’m going to try to set up all the examples on MyBinder, so stayed tuned for that!

[banesullivan]

Perhaps this is something that would make sense to have a reusable setup script? I’m thinking this might be something to put in https://github.com/fatiando/continuous-integration or maybe we should make our own repo of setup scripts related to viz?

What do you think @leouieda?

[banesullivan]

For now, I have made a section in the docs under installation on how to set up vtki for Docker containers/CIs with headless displays. See 44f3803 and http://docs.vtki.org/getting-started/installation.html#running-on-ci-services

[banesullivan]

Update: to make life even easier, I've created a cookie cutter to quickly make a MyBinder project: https://github.com/vtkiorg/cookiecutter-vtki-binder

And here is a sample project created using that template:

[akaszynski]

The sample binder looks awesome!

[banesullivan]

Once the dependencies for #144 get released, we can have fully interactive rendering on web examples like this!! That way people can get to using vtki by just opening a webpage - no need to install to test drive vtki!

[akaszynski]

It's going to be a game changer. Being able to have vtki run in a binder instance is already cool, but having it interactive means anyone will be able to interactively view vtki results. It's going to really bring it to the masses.

[banesullivan]

It's going to really bring it to the masses.

Hopefully, this gets it started:

Now all examples from the gallery are available on MyBinder 🎉 🎉

[daniellivingston]

Thank you so much! You've been so helpful.

I was having trouble propagating environment var DISPLAY to the subshell that Python lives in (it turns out it's a little tricky in Docker to do this...), but I settled on this as a working solution:

def activate_virtual_framebuffer():
    '''
    Activates a virtual (headless) framebuffer for rendering 3D
    scenes via VTK.

    Most critically, this function is useful when this code is being run
    in a Dockerized notebook, or over a server without X forwarding.

    * Requires the following packages:
      * `sudo apt-get install libgl1-mesa-dev xvfb`
    '''

    import subprocess
    import vtki

    vtki.OFFSCREEN = True
    os.environ['DISPLAY']=':99.0'

    commands = ['Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 &',
                'sleep 3',
                'exec "$@"']

    for command in commands:
        subprocess.call(command,shell=True)

And then, on module import, I check if I'm in the Docker container (in a rather inelegant way...) and run the function from there:

if os.path.isdir('/home/jovyan'):
    activate_virtual_framebuffer()

Thank you again!

[leouieda]

What do you think @leouieda?

@banesullivan that's a good point. I'm not sure. Is this issue exclusive to VTK? Are there fixes for Windows and Mac (not using containers)?

[leouieda]

Note that vtki will have to be used in offscreen mode. This can be forced on import

@banesullivan does this mean that the same code won't work to generate the docs and run in the user's computer? If so, I would recommend allowing this to be set through an environment variable as well so that no changes to the code are required.

[banesullivan]

Is this issue exclusive to VTK?

Actually no, the setup for this is well documented in the Travis docs and it’s for anything graphics related where a headless display is needed.

Are there fixes for Windows and Mac (not using containers)?

Windows has caused me many headaches in this regard... not sure. If running on a local machine with a display none of this is an issue

does this mean that the same code won't work to generate the docs and run in the user's computer?

We definitely overlooked this - yes.

Currently, we set vtki.OFFSCREEN in our conf.py... a user making their own docs would definitely get tripped up by this.

I think adding an environmental variable in the start script and having vtki look for that variable on import a much more appropriate solution. That way folks can use the a Travis Cookiecutter without any hurdles