make fails with xcb_renderutil.h

[SanskritFritz]

I tried to build x-choyce with make and got this result:

g++ -g -std=c++11 -Wall -O3 -I/usr/include/libdrm  -c x_client_thumbnail_gl.cpp
In file included from x_client_thumbnail_gl.cpp:4:0:
/usr/include/xcb/xcb_renderutil.h:61:39: error: expected ‘,’ or ‘...’ before ‘template’
     const xcb_render_pictforminfo_t  *template,
                                       ^
Makefile:43: recipe for target 'x_client_thumbnail_gl.o' failed
make: *** [x_client_thumbnail_gl.o] Error 1
make: *** Waiting for unfinished jobs....

What is the status of this project, is it usable? I stumbled into this accidentally, but I'm curious.

[jchnkl]

Hi,

many thanks for your interest!
You stumbled across this particular bug in xcb-renderutil.
However, the master branch does not use renderutil anymore, which means I've forgotten to remove the header. I've removed it already in the develop branch and now I cherry-picked the commit into master. Hence, it should build fine now.

Considering the status: master should (and hopefully is) always in a compile & usable state.
I'm using it almost from the first day I started working on it.

There currently a lot of changes in the develop branch, like:

• Icons
• Window names & titles
• Configuration through Xresource (without restarting)
• Handling of unmapped windows (this is special, because unmapped windows do not have a backing pixmap)
• Bugfixes, Performance improvements, etc.

If you're really curious, you can check out develop and glance at it.
However, documentation is not yet available, sorry.
tl;dr; check back, good things to come!

[SanskritFritz]

Hello, thanks for your answer. Now indeed I was able to build the project. However:
In the master branch: when I run it, I get a segfault as soon as I press alt-tab.
In the develop branch: when I run it, nothing happens upon alt-tab, it just prints some garbage onto the stdout.
I'm using Openbox without any compositing, that could be an issue?

UPDATE: tried it in KDE compositing, same result.

[jchnkl]

That's weird.
Compositing should not be the problem, the thumbnail windows use an ARGB visual, but without compositor they should just be opaque. I'm using openbox myself and tried out master again. Not working, seems like I broke something here with the latest push. Sorry.

However, in develop there is now a working version (I use it right now) which I'm very likely going to merge.

I changed the default key from mod1 to mod4 (that's most often the windows key), that's probably the reason for the garbage you see (Does it look like ^[?). However, it's configurable now. Add a line like this to your ~/.Xresources: XChoyce.mod: mod4 and reload with xrdb -merge ~/.Xresources.

It should work now. If you experience further problems, feel free to contact me again. :)

[SanskritFritz]

Thanks. I compiled develop again, and upon pressing super-tab I get this and the program exits:

X Error of failed request:  BadValue (integer parameter out of range for operation)
  Major opcode of failed request:  153 (GLX)
  Minor opcode of failed request:  16 (X_GLXVendorPrivate)
  Value in failed request:  0x20de
  Serial number of failed request:  0
  Current serial number in output stream:  296

[jchnkl]

Thanks for your feedback and patience!
It looks like that I've used some Vendor specific API functions.
What graphics card are you using (Intel, Nvidia, AMD)?

I'll look further into this. As far as I know there are at least three different versions for the OpenGL API. Vendor specific, proposed for the standard and the standard itself. Unfortunately OpenGL is rather complex, but I hope I can get this sorted out.

[jchnkl]

Ok, I pushed a commit to develop which could possible fix this for Nvidia GPUs. Since I don't have a Nvidia GPU, it would be very nice if you could try this out. Thanks!

[SanskritFritz]

Yes, I have nvidia GeForce 8600 GT using the proprietary driver (nouveau doesn't work here).
Well, we're getting there, now I was able to witness some grey boxes covering most of my screen which clearly corresponded to my visible windows, but then my linux simply locked up. Only the magic REISUB could unlock it.
I heard Mesa API is vendor independent, maybe you should use that?

[jchnkl]

I've checked the OpenGL implementation on my machine and it actually is the one provided by Mesa.
I used to have a Nvidia card myself (in fact even a 8600GT, too), and I think (iirc) that the binary driver replaced libgl with it's own version. Now I think it's well possible that this might be a driver bug. However without a Nvidia box, I cannot check this atm. Maybe I can look into getting my old card running again.

I'm not an OpenGL guru, I just put the bits and pieces together on the fly and I'm still learning. Currently I'm factoring out the OpenGL implementation to make it more abstract, so I can put in the XRender backend again.

Meanwhile, it would be very nice if you could tell me your driver version. It would be even more awesome if you could give nouveau a spin. ;)

Thanks for your help and feedback, it's much appreciated!

[SanskritFritz]

There is something definitely wrong with this nvidia card, because the newest drivers fail to work with it too, so this might be a problem. I'm currently using the last working version (319.32) of the nvidia driver, all newer versions failed to work here. I was never able to get nouveau working with this card. Time to buy a new video card? :)

[jchnkl]

I'm sorry to hear that. I hope you can get an appropriate replacement soon. It's a pity that nouveau doesn't work. It worked for me, but was terrible slow with rotated monitors.
Maybe I should just jump the gun and publicly announce this project in some forums to get more feedback from users. :)

[SanskritFritz]

Well, check out these archlinux packages:
https://aur.archlinux.org/packages/x-choyce-git/
https://aur.archlinux.org/packages/x-choyce-dev-git/
😄

About the nvidia problem of mine: you should know that apart from this lockup I actually don't have any issues with my card, Compiz, KDE, even Descent all work flawlessly with it.

[jchnkl]

Hey, thanks, I feel honored! :)
However, I honestly don't see how I could find a solution to this problem. It works absolutely flawless with my Intel GPU. I need to get that Nvidia card of mine back running to investigate this problem and that's going to take some time.
Do you have experience with gdb? A backtrace when it locks up could be helpful (although hard to obtain when the whole box hangs..). If you can switch to a virtual console, you could run gdb x:choyce in tmux and then obtain the backtrace on the vt. But only if it's not too much of a hassle. :)
That lock-up by itself is strange too: is your whole box unresponsive (i.e. no access via ssh, or vts) or just X?

[SanskritFritz]

You're welcome 😃
Ok, I'm not very familiar with gdb, but read up the usage, so I tried a short session. ssh access from my wife's laptop was still alive the time the program locked up, so I was able to press ctrl-c within an attached tmux session, and get a backtrace: http://sprunge.us/eheS
Is this what you wanted?
The lock up is sometimes the whole computer, sometimes only X. Through ssh-tmux I saw that the X process was taking up 100% CPU, when I killed it, I could login again locally.

[jchnkl]

Thanks, that's actually really useful. The problem is, that the my shaders don't get compiled (they are in the *.frag files).
It interesting that a warning results in a program build failure. Normally warnings don't mean failure.
I'll try to build a strip down version to isolate the issue. Stay tuned. ;)

[jchnkl]

So.. here we go.
If you're still in the mood you can try out this stuff:

1. In branch nogl you'll find a program without all the OpenGL stuff. When you hit Mod+Tab some oddly colored windows should appear, and you should be able to switch windows. Without indication though, just the "ok, it works" effect. It should be possible to repeatedly do so, without locking up X or crashing your computer.

2. In branch shaders you'll find a new directory tests. Change to that directory and type make. Then run the resulting binary: ./shaders. This is a simple test program which does nothing else than to compile glsl shaders. If you could show me the output, that'd be very much appreciated!

Thanks for your help and cheers!

[SanskritFritz]

1. Ok, it works.

GL_VENDOR:                   NVIDIA Corporation
GL_RENDERER:                 GeForce 8600 GT/PCIe/SSE2
GL_VERSION:                  3.3.0 NVIDIA 319.32
GL_SHADING_LANGUAGE_VERSION: 3.30 NVIDIA via Cg compiler

Compiling normal shader
No message means success

Compiling grayscale shader
No message means success

Compiling wrong shader
No message means success
Shader compilation for wrong failed:
0(8) : error C0000: syntax error, unexpected ')', expecting "::" at token ")"
0(12) : error C1048: invalid character 'n' in swizzle "n"

p creation for wrong failed:
Fragment info
-------------
0(8) : error C0000: syntax error, unexpected ')', expecting "::" at token ")"
0(12) : error C1048: invalid character 'n' in swizzle "n"

[jchnkl]

Thank you very much!
I've pushed some changes to the shaders branch. Would be nice if you could run it again.
If you're feeling adventurous you could try out the develop branch.
I think (from what I've seen) that it should work now. But keep that ssh shell around just to be safe. :)

[SanskritFritz]

Sadly I get the same lockup as before. I double checked it. No messages printed.

[jchnkl]

Thanks!
Honestly, I'm kind of clueless right now. That's weird, because it runs perfectly fine with my setup and you know, OpenGL should be platform independent.. :(

[jchnkl]

Ok, I think it all boils down to try and error here.
I've pushed branches narrow_1, narrow_2, narrow_3, narrow_4 and narrow_5.
1, 2 and 3 each try to isolate a single problem, 4 is a combination of 1, 2 and 3, 5 combines 1 & 2.
This removes functionality, especially 2 should not highlight any windows for selection.
However, choosing windows should still work (similar to the nogl branch).
If one of those branches works without lock up, were closer to the culprit. :)
If you do not want to help me on this issue anymore, that's not a problem. Don't feel obligated.

[jchnkl]

Btw.: Did you take a look at your X log files (/var/log/Xorg.0.log or similar), dmesg and /var/log/{messages,everything} or journalctl for error messages? Maybe there are some hints too.

[SanskritFritz]

I don't mind to test, I like to help 😄
Well, narrow_4 and narrow_5 are working, the others are not.
Nope, logs don't reveal anything.
BTW people in the archlinux community started to use your program 😃

[jchnkl]

That's awesome, your help is much appreciated. :)
I've pushed a commit to develop from which I hope that it solves the issue. Please run make clean before rebuilding, because I've changed a header file. Getting the compilation dependencies right is rather difficult, so cleaning is easier. ;)
That's really cool to have more users. I'm running arch myself and once this is stable I'm going to announce this in the arch forum.

[SanskritFritz]

Sadly this doesn't work either 😢

[jchnkl]

sigh
I'm coming slowly to my wits' end.
Well then.. narrow_6 and narrow_7 each remove some code which might cause trouble.

[SanskritFritz]

narrow_6 works, narrow_7 doesn't.

[jchnkl]

Cool, now we're getting somewhere. Thank you!
The problem is obviously the mipmap generation.
The call to glGenerateMipmap() should be supported by OpenGL 3.0, which in turn should be supported by your Nvidia card. Can I get the output of the glxinfo from computer, please?

There's a change in narrow_8 which could end this issue. I'm not to optimistic though, because all I did was to call glGenerateMipmap() earlier. The problem is, that I can't find much information about similar issues. Seems like everywhere else Nvidia + Mipmaps is doing fine. :)

However, I'm still curious about what's going on here. It's very odd that that your whole computer locks up. I suspect that the Nvidia driver causes either X to spin at 100% or even causes a kernel panic.
From what I've read, worst thing that should happen is either my application crashing or just a white texture.
The last thing we could resort to, would be to give up on Mipmaps entirely. This wouldn't be a functional loss, but a graphical (Mipmaps are necessary to make the scaling look prettier).

Oh, before I forget about: I change the build system a bit. Sources are now in src, and there's a make install target. To try it out run make SHADER_PATH=. from the top level directory and then the x:choyce binary within the src directory. (SHADER_PATH specifies the path to the *.frag files, which are, without installation in the src folder)

[jchnkl]

Ok, after I found this (with this) and this (with this), I've created narrow_9.
I'm slightly optimistic. :)

[SanskritFritz]

I'm very sorry to say but narrow_9 completely locked up everything again 😟

[jchnkl]

Alright, I've got my old box running again with nvidia-310.19-2.

GL_VENDOR:                   NVIDIA Corporation
GL_RENDERER:                 GeForce 8600 GT/PCIe/SSE2
GL_VERSION:                  3.3.0 NVIDIA 310.19
GL_SHADING_LANGUAGE_VERSION: 3.30 NVIDIA via Cg compiler

Now, I get the error message below from X. That's very interesting, makes me think of a driver bug again.

[Edit]
Reassurance. Even more. Also very relevant (possible fix)
(There's a reason why I hate Nvidia's drivers whole-heartedly. Nothing but problems, since I first bought these cards in 2008, using the binary driver on FreeBSD, sometimes slow, sometimes crashes.)
[/Edit]

(EE) [mi] EQ overflowing.  Additional events will be discarded until existing events are processed.
(EE)
(EE) Backtrace:
(EE) 0: X (xorg_backtrace+0x36) [0x58a426]
(EE) 1: X (mieqEnqueue+0x26b) [0x56b78b]
(EE) 2: X (0x400000+0x4c632) [0x44c632]
(EE) 3: /usr/lib/xorg/modules/input/evdev_drv.so (0x7f555e013000+0x5f74) [0x7f555e018f74]
(EE) 4: X (0x400000+0x75b97) [0x475b97]
(EE) 5: X (0x400000+0x9f258) [0x49f258]
(EE) 6: /usr/lib/libpthread.so.0 (0x7f5564550000+0xf1a0) [0x7f556455f1a0]
(EE) 7: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f555ea82000+0x7bd0b) [0x7f555eafdd0b]
(EE) 8: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f555ea82000+0x7d254) [0x7f555eaff254]
(EE) 9: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f555ea82000+0xf5356) [0x7f555eb77356]
(EE) 10: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f555ea82000+0x543ce8) [0x7f555efc5ce8]
(EE) 11: X (0x400000+0x17a6c3) [0x57a6c3]
(EE) 12: X (0x400000+0xc4ae0) [0x4c4ae0]
(EE) 13: X (0x400000+0x34fda) [0x434fda]
(EE) 14: X (0x400000+0x37e61) [0x437e61]
(EE) 15: X (0x400000+0x2696a) [0x42696a]
(EE) 16: /usr/lib/libc.so.6 (__libc_start_main+0xf5) [0x7f55631de725]
(EE) 17: X (0x400000+0x26cad) [0x426cad]
(EE)
(EE) [mi] These backtraces from mieqEnqueue may point to a culprit higher up the stack.
(EE) [mi] mieq is *NOT* the cause.  It is a victim.
(EE) [mi] EQ overflow continuing.  100 events have been dropped.
(EE)
(EE) Backtrace:
(EE) 0: X (xorg_backtrace+0x36) [0x58a426]
(EE) 1: X (0x400000+0x4c632) [0x44c632]
(EE) 2: /usr/lib/xorg/modules/input/evdev_drv.so (0x7f555e013000+0x5f74) [0x7f555e018f74]
(EE) 3: X (0x400000+0x75b97) [0x475b97]
(EE) 4: X (0x400000+0x9f258) [0x49f258]
(EE) 5: /usr/lib/libpthread.so.0 (0x7f5564550000+0xf1a0) [0x7f556455f1a0]
(EE) 6: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f555ea82000+0x7bd0b) [0x7f555eafdd0b]
(EE) 7: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f555ea82000+0x7d254) [0x7f555eaff254]
(EE) 8: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f555ea82000+0xf5356) [0x7f555eb77356]
(EE) 9: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f555ea82000+0x543ce8) [0x7f555efc5ce8]
(EE) 10: X (0x400000+0x17a6c3) [0x57a6c3]
(EE) 11: X (0x400000+0xc4ae0) [0x4c4ae0]
(EE) 12: X (0x400000+0x34fda) [0x434fda]
(EE) 13: X (0x400000+0x37e61) [0x437e61]
(EE) 14: X (0x400000+0x2696a) [0x42696a]
(EE) 15: /usr/lib/libc.so.6 (__libc_start_main+0xf5) [0x7f55631de725]
(EE) 16: X (0x400000+0x26cad) [0x426cad]
(EE)
[mi] Increasing EQ size to 512 to prevent dropped events.
[mi] EQ processing has resumed after 169 dropped events.
[mi] This may be caused my a misbehaving driver monopolizing the server's resources.

[jchnkl]

I'm banging my head against the table here. Something is severely wrong with either my shaders, my program, Intel's GL implementation or Nvidia's GL implementation.
I worked around the driver bug from above, but just to find out that my shader (which works just the way I want on my Intel laptop) produces a completely different result on the Nvidia card.
I should have written that damn thing with OpenCL from the beginning.. :/

[jchnkl]

So, after much frustration I replaced the whole fixed pipeline with a vertex and a fragment shader.
Now it works nice on Nvidia too. However, the fonts are not drawn properly. I'm going to replace xft with either ftgl or QuesoGLC, not sure yet.
And there's a problem with compton which I don't understand. I don't even know if it's a problem with compton and/or nvidia or my application. sigh

[SanskritFritz]

So that means, some dependencies have changed? Which branch should I try now? Also, if you want to take over the AUR packages, just tell me.

[jchnkl]

Alright.

Glad to tell that I've fixed the mipmap issue (which brought my focus to other issues as well).
In particular, I'm now choosing the proper FBConfig for transparent windows (for compositing), hence it now works like intended with e.g. compton.
I've also fixed titles with xft, i.e. no need for additional dependencies.

Now, these were all issues which weren't obvious or did not appear on my laptop with Intel gfx. Bloody mess, talk about standards an platform independence. Don't know whose to blame here, but from what I read it's Intel who isn't following the standards. Now I just hope that this will work without problems for folks with ATI cards.

Enough ranting. :)
If you want to check out the latest and greatest, try the nvidia branch. I am up to 99% sure, that it won't crash your box. (At least, it doesn't crash mine..).

That's my messy development branch atm, for syncing between my development machine and my nvidia test box. Not suitable for publishing. ;) I'll clean up and write commits later, then I'll push to master.

About AUR: If you do not want to administer the packages anymore, then just disown and I'll pick them up. Otherwise, I am fine with the current situation.

[SanskritFritz]

I tried the nvidia branch, what's the hotkey?
I'm glad to help with the AUR, so let's continue with the current situation.

[jchnkl]

Ah, sorry, for testing I've changed it temporarily back to Alt-Tab again, so it wouldn't interfere with my running setup.

[jchnkl]

If you feel lucky, you can give the develop branch a try. Go into src and type make; make clean; ./x:choyce. Should work out of the box. Tested on my Intel and Nvidia boxes.

[SanskritFritz]

develop spits out some errors:

Program creation for grayscale_shader failed:
Vertex info
-----------
(0) : error C5145: must write to gl_Position

otherwise it works :)

[jchnkl]

Hm, I just pushed some changes (maybe 2 or 3 minutes ago). I also tried that latest revision just now, and it works for me. Maybe you could try again?
Otherwise, I'm glad to hear that it works now for you too!

[SanskritFritz]

Ah, now it's working, nice! Thanks for your patient work.

[jchnkl]

I'm very delighted to hear that!
I value your contribution and help very high (actually, this reminds me, that I wanted to add a "Contributors" section in the readme). If we ever meet in person, I owe you a beer or whatever you're after. ;)

I'm going to create another AUR package based on https://github.com/jrk-/x-choyce/archive/master.zip and add it to the sources. I hope this will be more appealing for people who do not {have,want} git installed.

[jchnkl]

Here you go.
I didn't note it explicitly (and I should probably add a LICENSE file) but I intend to release this under the 3-clause BSD license. :)

[SanskritFritz]

I think you should create explicit tags and version numbers, so that a stable version will be guaranteed in the package.

[jchnkl]

Great idea, thanks! I'm using date -I as tags and modified the PKGBUILD accordingly.
Now version 2013.12.09 corresponds to the tag 2013-12-09.
[Edit] Except that AUR doesn't like the bash expansion for pkgver. But I think I leave it like that for now..

[SanskritFritz]

That's right, nice! Well, version is not needed, you have pkgver for that, and the AUR will be happy too 😄

[SanskritFritz]

Also, packaging standards recommend using md5sums.

[jchnkl]

Hm, the point is, that I'd like to use date -I (ISO date) as tags. But AUR doesn't allow dashes (-) in version numbers. Maybe I'll set up a git hook for tagging or so.
About md5sums: I used to try this with gitweb tarballs, but those where not reproducible, resulting in different md5 sums for every download. Let's see if github can do better.