background generated in front with headless egl

[MarcoP91]

Hello,

I am trying to replicate the results of this paper, which created a mesh from a 2d image and then uses vispy to create a camera in order to move around the mesh and make a video. On my local ubuntu machine with PyQt5 backend and by using the method _screenshot() to take the frames from the video, everything works fine.

vispy.sys_info output:
Platform: Linux-5.4.0-42-generic-x86_64-with-debian-buster-sid Python: 3.7.7 (default, Mar 26 2020, 15:48:22) [GCC 7.3.0] NumPy: 1.18.5 Backend: PyQt5 pyqt4: None pyqt5: ('PyQt5', '5.9.2', '5.9.6') pyside: None pyside2: None pyglet: None glfw: None sdl2: None wx: None egl: None osmesa: None _test: None

video output:

However, when I try to do the same on my headless gcloud instance, I can not use PyQt5 because of this:

WARNING: QXcbConnection: Could not connect to display
WARNING: Could not connect to any X display.

By using the egl backend, everything works, except that the background appears to generate in front of the foreground.

sys info:
Platform: Linux-4.9.0-11-amd64-x86_64-with-debian-9.11
Python: 3.7.7 (default, Mar 26 2020, 15:48:22) [GCC 7.3.0]
NumPy: 1.18.5
Backend: egl
pyqt4: None
pyqt5: ('PyQt5', '5.9.2', '5.9.6')
pyside: None
pyside2: None
pyglet: None
glfw: None
sdl2: None
wx: None
egl: EGL 1.5 NVIDIA: OpenGL_ES OpenGL
osmesa: OSMesa
_test: None
GL version: '4.6.0 NVIDIA 418.87.00'

output:

The code is the same in the 2 versions, the background is the only thing that changed.

Could I please have some suggestions on how this happens and how could I solve it?

[djhoese]

I'm not sure what's up with the EGL backend. That's pretty weird. I don't have any experience with EGL so don't think I can help.

For PyQt5, you'll need to look at creating a virtual display. You can see how we do that in our own Travis CI testing:

vispy/.travis.yml

Lines 135 to 138 in bb2cd96

	export DISPLAY=:99.0;
	if [ "${TRAVIS_OS_NAME}" = "osx" ]; then ( sudo Xvfb :99 -ac -screen 0 1400x900x24 +render +iglx; echo ok )& fi;
	if [ "${TRAVIS_OS_NAME}" = "linux" ]; then
	/sbin/start-stop-daemon --start --quiet --pidfile /tmp/custom_xvfb_99.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :99 -screen 0 1400x900x24 -ac +extension GLX +render;

The important part are the export and the Xvfb calls.

[MarcoP91]

I included the xfvb call as you suggested, but it still did not work. I have ended up making it work by uninstalling my nvidia drivers. However, without those drivers I cannot use pytorch and run the models on the GPU. I tried reinstalling the drivers, but this error (the same that I had previously) appears:
WARNING: QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled WARNING: QXcbIntegration: Cannot create platform offscreen surface, neither GLX nor EGL are enabled WARNING: QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled WARNING: QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled WARNING: QOpenGLWidget: Failed to create context WARNING: QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled WARNING: QOpenGLWidget: Failed to create context
These warnings are followed by a canvas.render error:
WARNING: Error drawing visual <Mesh at 0x7f9478698d50> 0%| | 0/1 [01:16<?, ?it/s] Traceback (most recent call last): File "main.py", line 211, in <module> depth_after_range = depth_after_range, focus_val = focus_val) File "/home/marco/3d-photo-inpainting/mesh_turntable.py", line 2468, in output_3d_photo img = normal_canvas.render() File "/home/marco/3d-photo-inpainting/mesh_turntable.py", line 2401, in render return self.canvas.render() File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/scene/canvas.py", line 257, in render self._draw_scene(bgcolor=bgcolor) File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/scene/canvas.py", line 266, in _draw_scen e self.draw_visual(self.scene) File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/scene/canvas.py", line 304, in draw_visua l node.draw() File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/scene/visuals.py", line 99, in draw self._visual_superclass.draw(self) File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/visuals/mesh.py", line 519, in draw Visual.draw(self, *args, **kwds) File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/visuals/visual.py", line 443, in draw self._vshare.index_buffer) File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/visuals/shaders/program.py", line 101, in draw Program.draw(self, *args, **kwargs) File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/gloo/program.py", line 533, in draw canvas.context.flush_commands() File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/gloo/context.py", line 175, in flush_comm ands self.shared.parser.parse([('CURRENT', 0, fbo)]) File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/gloo/glir.py", line 819, in parse self._parse(command) File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/gloo/glir.py", line 743, in _parse self._gl_initialize() File "/home/marco/miniconda3/envs/3DP/lib/python3.7/site-packages/vispy/gloo/glir.py", line 851, in _gl_initializ e if this_version < '2.1': File "/home/marco/miniconda3/envs/3DP/lib/python3.7/distutils/version.py", line 52, in __lt__ c = self._cmp(other) File "/home/marco/miniconda3/envs/3DP/lib/python3.7/distutils/version.py", line 335, in _cmp if self.version == other.version: AttributeError: 'LooseVersion' object has no attribute 'version'

Now, whether I set the program to use the GPU or run on CPU only, this same error appears. Do you think there is a way to make pyqt5 work while keeping the nvidia drivers? I have looked at this thread #1503 and tried to create a conda env with pyqt<5, but this generates a lot of conflicts with the requirements that the project has (and apparenly it takes 24h to even identify them) Are there some others workarounds?

Some infos:

System:
Linux tensorflow-1-vm 4.9.0-13-amd64 #1 SMP Debian 4.9.228-1 (2020-07-05) x86_64 GNU/Linux
Nvidia driver:
NVIDIA-SMI 440.95.01 Driver Version: 440.95.01 CUDA Version: 10.2

[djhoese]

You ran xvfb but did you also add the export DISPLAY line? That line tells X to draw things to the display that xvfb is creating (:99). If that doesn't work then you probably need to install GLX so that X can talk to OpenGL. At least that's my understanding.

[MarcoP91]

That was not the problem, as I was running every command with DISPLAY=:99 at the beginning. I tried many things but in the end I was not able to make it work on Debian 9. I created a new instance with Ubuntu 18.04, reinstalled everything, added the NVIDIA drivers and now it works both on cpu/gpu. Everything should be ok now, thank you, I will close the issue.