Archlinux Intel Driver Crashes

[like-a-bause]

Hi I am sitting on an updated Archlinux using current OF Master on a MacBookPro 11,1.
On some calls (e.g. gl_renderer_->drawAxis(10);) the Application crashes with a segfault.
I tried some examples. alphaMaskingShaderExample works, areaLightExample crashes.
The bt for alphaMaskinShader:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff4129355 in __memcpy_avx_unaligned () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff4129355 in __memcpy_avx_unaligned () from /usr/lib/libc.so.6
#1  0x00007fffedfa3886 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#2  0x00007fffedfa4157 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#3  0x00007fffedfa4386 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#4  0x00007fffee013304 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#5  0x00007fffedfa28e4 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#6  0x00007fffede04e4c in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#7  0x000000000047eb25 in ofGLProgrammableRenderer::drawInstanced(ofVboMesh const&, ofPolyRenderMode, int) const ()
#8  0x000000000047da9c in ofGLProgrammableRenderer::draw(of3dPrimitive const&, ofPolyRenderMode) const ()
#9  0x0000000000431883 in of3dGraphics::drawAxis(float) const ()
#10 0x000000000047d8bc in ofGLProgrammableRenderer::draw(ofNode const&) const
    ()
#11 0x0000000000424c86 in ofApp::draw() ()
#12 0x0000000000557a64 in Poco::PriorityDelegate<ofBaseApp, ofEventArgs, false>::notify(void const*, ofEventArgs&) ()
#13 0x00000000004c2a79 in Poco::PriorityStrategy<ofEventArgs, Poco::AbstractPriorityDelegate<ofEventArgs> >::notify(void const*, ofEventArgs&) ()
#14 0x00000000004c61e7 in void ofNotifyEvent<ofEvent<ofEventArgs>, ofEventArgs>(ofEvent<ofEventArgs>&, ofEventArgs&) ()
#15 0x00000000004c0c1a in ofCoreEvents::notifyDraw() ()
#16 0x000000000055be06 in ofAppGLFWWindow::draw() ()

I am trying to be as verbose as possible about my current Packages:

Linux megalocerus 3.19.3-3-ARCH #1 SMP PREEMPT Wed Apr 8 14:10:00 CEST 2015 x86_64 GNU/Linux
mesa 10.5.3-1
xf86-video-intel 2.99.917-5
xorg-server 1.17.1-5

Output of glxInfo:

OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) Haswell 
OpenGL core profile version string: 3.3 (Core Profile) Mesa 10.5.3
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 3.0 Mesa 10.5.3
OpenGL shading language version string: 1.30
OpenGL context flags: (none)
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.0 Mesa 10.5.3
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.00
OpenGL ES profile extensions:

Any hints on further debugging/fixing?

[bilderbuchi]

@openframeworks/linux?

[like-a-bause]

I made an apitrace trace and the last GL Call is:

3484 @0 glDrawArrays(mode = GL_LINES, first = 0, count = 6)
caught an unhandled exception glretrace: signalHandler+0x23655c: /home/felix/Source/apitrace/common/os_posix.cpp:324 /usr/lib/libpthread.so.0+0x1073f /usr/lib/libc.so.6+0x127355 /usr/lib/xorg/modules/dri/i965_dri.so+0x315885 /usr/lib/xorg/modules/dri/i965_dri.so+0x316156 /usr/lib/xorg/modules/dri/i965_dri.so+0x316385 /usr/lib/xorg/modules/dri/i965_dri.so+0x385303 /usr/lib/xorg/modules/dri/i965_dri.so+0x3148e3 /usr/lib/xorg/modules/dri/i965_dri.so+0x176e4b glretrace: retrace_glDrawArrays+0xc7549: /home/felix/Source/apitrace/build/retrace/glretrace_gl.cpp:7721 glretrace: retraceCall+0xbbf8: /home/felix/Source/apitrace/retrace/retrace_main.cpp:209 glretrace: _ZN7retrace11RelayRunner6runLegEPN5trace4CallE+0xc2d9: /home/felix/Source/apitrace/retrace/retrace_main.cpp:366 glretrace: _ZN7retrace11RelayRunner7runRaceEv+0xc2d9: /home/felix/Source/apitrace/retrace/retrace_main.cpp:337 glretrace: _ZN7retrace9RelayRace3runEv+0xc2d9: /home/felix/Source/apitrace/retrace/retrace_main.cpp:506 glretrace: mainLoop+0x7a54: /home/felix/Source/apitrace/retrace/retrace_main.cpp:566 glretrace: main+0x7a54: /home/felix/Source/apitrace/retrace/retrace_main.cpp:837 /usr/lib/libc.so.6: __libc_start_main+0xef glretrace: _start+0x28 ? apitrace: info: taking default action for signal 11

There are several Bug symbols in the Qt Gui. In frame 0 we got:

416 @0 glClear(mask = GL_COLOR_BUFFER_BIT)
shader compiler issue 2: VS vec4 shader: 6 inst, 0 loops, compacted 96 to 96 bytes.

Then in Frame 1 several glLinkProgram calls:

3104 @0 glLinkProgram(program = 1)
shader compiler issue 2: VS vec4 shader: 17 inst, 0 loops, compacted 272 to 256 bytes.

and then errors on glCreateBuffers, glNamedBufferData, and glBindBuffer, all "GL_INVALID_OPERATION"

3201 @0 glCreateBuffers(n = 1, buffers = [0])
glGetError(glCreateBuffers) = GL_INVALID_OPERATION

Open source graphics drivers on Linux are not good, especially for Intel hardware, if fact so awful that Valve made their own Intel driver for Vulkan already and it'll be open sourced when spec will be sorted out.

If you're doing a personal project and you need to get job done as fast as possible I'd recommend to switch to Nvidia or AMD hardware with proprietary drivers, but I've heard Mesa developers listen moans of users somewhere on their IRC channels and can fix bugs quickly.

[like-a-bause]

thanks @procedural
unfortunately switching to amd/nvidia isn't an option.
I asked on the #intel-gfx channel on freenode and they said this is a mesa bug. I compiled mesa with --enable-debug and installed. Now I get:

Mesa: User error: GL_INVALID_ENUM in glGetString(GL_EXTENSIONS)
Mesa: User error: GL_INVALID_OPERATION in unsupported function called (unsupported extension or deprecated function?)
Mesa: User error: GL_INVALID_OPERATION in unsupported function called (unsupported extension or deprecated function?)

This could indicate an error on OF side.
The bt still shows no verbose output for the driver so I can't file a bug report at mesa:

#1  0x00007fffedf354f6 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#2  0x00007fffedf36277 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#3  0x00007fffedf36586 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#4  0x00007fffedfb4e13 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#5  0x00007fffedf345f4 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so
#6  0x00007fffedd878d3 in ?? () from /usr/lib/xorg/modules/dri/i965_dri.so

Has anyone advice on how to compile for Debug Symbols?

@aspeteRakete http://www.mesa3d.org/debugging.html

More extensive error checking is done when Mesa is compiled with the DEBUG symbol defined. You'll have to edit the Make-config file and add -DDEBUG to the CFLAGS line for your system configuration. You may also want to replace any optimization flags with the -g flag so you can use your debugger. After you've edited Make-config type 'make clean' before recompiling.

Also, try to export these variables before running an app:

export LIBGL_DEBUG=verbose
export MESA_DEBUG=1
export EGL_LOG_LEVEL=debug

You can compile openFrameworks with debug symbols as well by adding -g compile flag to makefiles so you could see the exact line in drawInstanced() function before entering the driver.

I have debug symbols in CMake-based OF enabled by default for debug builds, btw :)

[arturoc]

just to say that all those examples are working for me fine on intel haswell on ubuntu with mesa 10.5.2.

also debug is active in the default makefiles when using the Debug target by calling make Debug or using the debug target in codeblcoks

@arturoc can it be possible that instancing is not supported on older Intel GPUs?

@aspeteRakete which videocard does your Macbook have? Can you upload full log of glxinfo?

[like-a-bause]

Its a late 2013 MBP so Intel Iris on board.
The full glxinfo output can be found here:
https://gist.github.com/aspeteRakete/e4d703b366cc26790466
Its not an old GPU. The Mac Driver supports GL 4.1
@arturoc yeah this never occured on an ubuntu box. I just updated to Mesa 10.5.4. Doesn't fix the problem.
@procedural I compiled Mesa with -DDEBUG and -g. Still no verbose output in gdb. I will have to try the env. vars later.

@aspeteRakete never compiled Mesa myself, so I can suggest the last "secret" trick I use for compiling stuff as I want and not as Make or other build system makes me want it... :) Not sure how exactly do this for autogen or makefiles, but try to call ./autogen --verbose so you would get the exact compile and link commands which happen during library assembly and then find the command that links i965_dri.so, on that line it will show all the object files which were linked to it and by their names you can figure out which sources were compiled -- copy and paste their compile lines and link command you found earlier to an empty bash script and add -g flag manually to each compile line.

[like-a-bause]

Now i got it. archlinux makepkg was stripping the Symbols.

#0  0x00007ffff4129355 in __memcpy_avx_unaligned () from /usr/lib/libc.so.6
#1  0x00007fffedf354f6 in memcpy (__len=72, __src=0x0, __dest=<optimized out>) at /usr/include/bits/string3.h:53
#2  copy_array_to_vbo_array (brw=brw@entry=0x7ffff7f7d040, min=min@entry=0, max=max@entry=5, buffer=buffer@entry=0x7ffff7fa0ea0, 
    dst_stride=12, element=0x7ffff7fa0da0) at brw_draw_upload.c:377
#3  0x00007fffedf36277 in brw_prepare_vertices (brw=0x7ffff7f7d040) at brw_draw_upload.c:572
#4  0x00007fffedf36586 in brw_emit_vertices (brw=0x7ffff7f7d040) at brw_draw_upload.c:652
#5  0x00007fffedfb4e13 in brw_upload_state (brw=0x7ffff7f7d040) at brw_state_upload.c:648
#6  0x00007fffedf345f4 in brw_try_draw_prims (indirect=<optimized out>, max_index=<optimized out>, min_index=<optimized out>, 
    ib=<optimized out>, nr_prims=<optimized out>, prims=<optimized out>, arrays=<optimized out>, ctx=<optimized out>) at brw_draw.c:514
#7  brw_draw_prims (ctx=0x7ffff7f7d040, prims=0x7fffffffdf00, nr_prims=72, ib=0x0, index_bounds_valid=168 '\250', min_index=0, 
    max_index=5, unused_tfb_object=0x0, indirect=0x0) at brw_draw.c:607
#8  0x00007fffedd878d3 in vbo_draw_arrays (ctx=0x7ffff7f7d040, mode=1, start=0, count=6, numInstances=1, baseInstance=0)
    at vbo/vbo_exec_array.c:645
#9  0x000000000046b91f in ofGLProgrammableRenderer::draw (this=0x11ad700, vbo=..., drawMode=1, first=0, total=6)
    at ../../../libs/openFrameworks/gl/ofGLProgrammableRenderer.cpp:426

Filed a bug report at mesa: https://bugs.freedesktop.org/show_bug.cgi?id=90177

@arturoc i'll try to compile mesa 10.5.2 and look if the problem persists.

[like-a-bause]

Same thing on 10.5.2