Difficulty installing with nouveau + nvidia

[edahlgren]

From this comment, a longer explanation of what I believe was my underlying issue installing and using nixGL. Assume Ubuntu 16.04 for apt commands / packages.

---

1. First I checked my configuration and found a "nouveau" driver being used for my NVIDIA GM108M [GeForce 940MX]. It looked something like this:

$ sudo lshw -c video
  *-display               
       description: 3D controller
       product: GM108M [GeForce 940MX]
       vendor: NVIDIA Corporation
       ...
       configuration: driver=nouveau latency=0

2. My host glxinfo showed that Intel Mesa was being used for GL. Unfortunately I don't still have these logs, but it looked something like this:

$ glxinfo | grep render
direct rendering: Yes
OpenGL renderer string: Mesa DRI Intel(R) 945GM GEM 20090326 2009Q1 RC2 x86/MMX/SSE2

3. I therefore assumed (perhaps wrongly, I don't know) that I needed to use nixGLIntel because the README.md says: "For mesa (intel, amd, nouveau, ...)". So I did:

$ git clone https://github.com/guibou/nixGL
$ cd nixGL
$ nix-build -A nixGLIntel
$ nix-env -i ./result

4. Then I entered a nix shell and ran glxinfo and it failed:

$ nix-shell -p glxinfo
[nix-shell:~]$ nixGLIntel glxinfo
glxinfo Error: couldn't find RGB GLX visual or fbconfig

5. So I strace'd glxinfo and discovered that it couldn't find nvidia shared libraries. I also noticed that in a last ditch attempt glxinfo looked for libGLX_indirect.so.0, couldn't find it, and failed.

---

At this point I was confused about whether I needed to use nixGLNvidia. So I started looking around for my exact nVidia version number (e.g. 390.25). Unfortunately I could only find the incomplete version string 384, so I downloaded nvidia-smi to discover an exact version. This is where things got weird.

$ sudo apt-get install nvidia-smi
$ nvidia-smi

This actually fetched (and apparently installed --- unintentional by me), a new nvidia 384.130 driver, but since I didn't reboot, lshw continued to show nouveau. I then used this version number (384.130) to build nixGLNvidia. I tested it, and it failed in the same way as before:

$ nix-shell -p glxinfo
[nix-shell:~]$ nixGLNvidia glxinfo
glxinfo Error: couldn't find RGB GLX visual or fbconfig

---

At this point it felt like nixGL wouldn't work at all for my setup. So I started looking into redirecting libGL through X --- an approach that I could reason about easily, at least more than than nvidia drivers --- which I didn't understand at all.

Through experimenting with my X server, I ended up rebooting. At that point the nvidia drivers became active (overriding nouveau), and lshw produced this:

$ sudo lshw -c video | grep 'configuration'
       configuration: driver=nvidia latency=0
       configuration: driver=i915 latency=0

While writing up this issue, I rebuilt nixGLNvidia, installed it, and tested it again with glxinfo to reproduce the error. This time direct rendering worked:

[nix-shell:~]$ glxinfo
name of display: :0
Error: couldn't find RGB GLX visual or fbconfig

[nix-shell:~]$ nixGLNvidia glxinfo | head
name of display: :0
display: :0  screen: 0
direct rendering: Yes
server glx vendor string: NVIDIA Corporation
server glx version string: 1.4
server glx extensions:
    GLX_ARB_context_flush_control, GLX_ARB_create_context, 
    GLX_ARB_create_context_profile, GLX_ARB_create_context_robustness, 
    GLX_ARB_fbconfig_float, GLX_ARB_multisample, GLX_EXT_buffer_age, 
    GLX_EXT_create_context_es2_profile, GLX_EXT_create_context_es_profile, 

---

Thoughts:

1. nixGLNvidia seems to work just fine for my configuration if nvidia drivers are being used, not nouveau. nixGLIntel didn't work with my nouveau driver and was the root cause of my confusion.

2. I wouldn't have even got nixGLNvidia working if I hadn't accidentally installed new nvidia drivers using nvidia-smi and happened to reboot (unrelated).

3. nixGL as a solution is great when it works (it's very nice to have fully-functioning direct rendering from a nix shell), but I ended up abandoning it because (a) I didn't know which nixGLXXX I was supposed to use, and (b) I didn't have the patience to debug it further when both nixGLIntel and nixGLNvidia didn't work for me.

So maybe in my case (and for others like me), the solution is to either provide scripts to fully detect which nixGLXXX to use, and automatically blacklist nouveau if it's known to cause problems, with a shell.nix to use (avoiding needing to remember to wrap binaries). If not all configurations can be covered/tested, then maybe some sort of fallback solution to indirect rendering could be suggested, or a fallback to disable hardware acceleration entirely (e.g. LIBGL_ALWAYS_SOFTWARE), just so the user can get something to work.

Hope this helps! 😃
Thanks for developing this!

[guibou]

Hello. Thank you for reporting this issue.

With nouveau you should use nixGLIntel (Which should definitly be renamed nixGLMesa). If you use the proprietary nvidia driver, you should use nixGLNvidia.

However, you appear to have an hybrid configuration, with an intel card AND a discreet nvidia GPU which is setup by default to use nouveau. I'm not used to this configuration (most people with hybrid configuration want to use the nvidia driver). However I'm impressed that you were able to get it work with nixGLNvidia considering that for hybrid hardware you should use nixGLNvidiaBumblebee.

However, nixGLIntel should work out of the box with nouveau. I'll experiment today with a computer with nouveau drivers and I'll try to see what happen.

[edahlgren]

Cool! Look forward to hearing what you find out.

[masaeedu]

I'm running into what I think is a similar situation (sorry if I'm conflating different issues): on Arch Linux I have the nouveau kernel module loaded, and the nvidia package uninstalled entirely. The device has an GTX 1060 graphics card, which nouveau picks up as code name NV136:

➜  snap git:(master) ✗ nixGLIntel glxinfo | grep render
direct rendering: Yes
    GLX_MESA_multithread_makecurrent, GLX_MESA_query_renderer,
    GLX_MESA_query_renderer, GLX_MESA_swap_control, GLX_OML_swap_method,
Extended renderer info (GLX_MESA_query_renderer):
OpenGL renderer string: NV136
    GL_ARB_conditional_render_inverted, GL_ARB_conservative_depth,
    GL_MESA_texture_signed_rgba, GL_NV_conditional_render,
    GL_ARB_compute_variable_group_size, GL_ARB_conditional_render_inverted,
    GL_NV_conditional_render, GL_NV_conservative_raster,
    GL_EXT_render_snorm, GL_EXT_robustness, GL_EXT_sRGB_write_control,
    GL_MESA_shader_integer_functions, GL_NV_conditional_render,
    GL_OES_element_index_uint, GL_OES_fbo_render_mipmap,

Now when I try to run a Haskell application that depends on gloss (using the GLUT backend), I get the following error:

➜  snap git:(master) ✗ nixGLIntel ./dist-newstyle/build/x86_64-linux/ghc-8.6.5/snap-0.0.0/x/snap/build/snap/snap
freeglut (snap):  ERROR:  Internal error <FBConfig with necessary capabilities not found> in function fgOpenWindow

When I strace this to investigate further, I see something like the following towards the end:

openat(AT_FDCWD, "/nix/store/n2cl0b3hbr6l0z1hxzwwkkwlndq5zq2r-mesa-20.0.2/lib/libxshmfence.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/7d40r6g9wjqndnmjk34xgzhd5rbf4lqs-expat-2.2.8/lib/libxshmfence.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/1swigvl8irvkwzdf23i060yz8nyi2ba2-libxshmfence-1.3/lib/libxshmfence.so.1", O_RDONLY|O_CLOEXEC) = 4
read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\20\0\0\0\0\0\0"..., 832) = 832
fstat(4, {st_mode=S_IFREG|0555, st_size=16592, ...}) = 0
mmap(NULL, 16504, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7f03ad366000
mmap(0x7f03ad367000, 4096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x1000) = 0x7f03ad367000
mmap(0x7f03ad368000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x2000) = 0x7f03ad368000
mmap(0x7f03ad369000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x2000) = 0x7f03ad369000
close(4)                                = 0
munmap(0x7f03ad432000, 542440)          = 0
munmap(0x7f03ad3f9000, 231672)          = 0
munmap(0x7f03ad3e4000, 84496)           = 0
munmap(0x7f03ad3c6000, 118904)          = 0
munmap(0x7f03ad3c1000, 16424)           = 0
munmap(0x7f03ad3ba000, 24680)           = 0
munmap(0x7f03ad3ad000, 16552)           = 0
munmap(0x7f03ad3b2000, 28952)           = 0
munmap(0x7f03ad380000, 180240)          = 0
munmap(0x7f03ad37a000, 20616)           = 0
munmap(0x7f03ad375000, 16472)           = 0
munmap(0x7f03ad36b000, 37048)           = 0
munmap(0x7f03ad366000, 16504)           = 0
openat(AT_FDCWD, "/nix/store/rpnax7fhi4np89fpzlcf2s79mfx458q6-mesa-20.0.2-drivers/lib/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/xzh4ngyyghhn6rby1x24p1qmmq6k6vch-mesa-20.0.2-drivers/lib/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/haswell/x86_64/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/haswell/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/x86_64/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "haswell/x86_64/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "haswell/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "x86_64/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/b2pj7aw0bk1mlchw4wkzlw83dxms2925-libX11-1.6.7/lib/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/rayy7mldlrrz4s88gkigkbsl3x4fswhz-libXext-1.3.3/lib/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/90v4pqq4wdfwcg98khqv7404rlh5hnnv-libglvnd-1.0.0/lib/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/h0p0h3rh1q4i2yavzm3yqi716s9yaj2f-glibc-2.27/lib/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/h0p0h3rh1q4i2yavzm3yqi716s9yaj2f-glibc-2.27/lib/libGLX_indirect.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "freeglut ", 9freeglut )                = 9
write(2, "(snap): ", 8(snap): )                 = 8
write(2, " ERROR:  Internal error <FBConfi"..., 97 ERROR:  Internal error <FBConfig with necessary capabilities not found> in function fgOpenWindow) = 97
write(2, "\n", 1
)                       = 1
poll([{fd=3, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=3, revents=POLLOUT}])
writev(3, [{iov_base="p\0\1\0<\0\2\0\0\0 \10+I\1\0", iov_len=16}, {iov_base=NULL, iov_len=0}, {iov_base="", iov_len=0}], 3) = 16
poll([{fd=3, events=POLLIN}], 1, -1)    = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\1\1\32\0\0\0\0\0\7\0@\3\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 32
shutdown(3, SHUT_RDWR)                  = 0
close(3)                                = 0
getpid()                                = 60754
exit_group(1)                           = ?
+++ exited with 1 +++

i.e. just as @edahlgren had observed, the problem is the inability to resolve libGLX_indirect.so.0. It isn't super clear to me where this library is supposed to be found. Would you have some insight on the matter?

[guibou]

@masaeedu thank you for the detailled report.

To be sure, you glxinfo is the one from nixpkgs or does it comes from your host system?

Is your haskell application built in a nix environment? Using GHC from nix and with nix dependencies? Because if not, perhaps it is not needed to run it under nixGL.

I do realize that "I'm experimenting today with a computer with nouveau driver" was one year old. Sorry about that @edahlgren