NVIDIA/Xorg segfaults with hardened profile

[ivan]

Issue description

With the hardened profile imported, Xorg on NVIDIA always segfaults on startup with:

[    43.702] (EE) Backtrace:
[    43.703] (EE) 0: /nix/store/aq6widncxzcr8kndsrr42a9c61r39qll-xorg-server-1.20.4/bin/X (OsSigHandler+0x29) [0x59af79]
[    43.703] (EE) 1: /nix/store/xqs95fqkjb1kd102yjv5h5q57gcsafb3-glibc-2.27/lib/libpthread.so.0 (funlockfile+0x50) [0x7bdefc8598af]
[    43.704] (EE) 2: /nix/store/xqs95fqkjb1kd102yjv5h5q57gcsafb3-glibc-2.27/lib/libc.so.6 (__memmove_avx_unaligned_erms+0x211) [0x7bdefc7e3bf1]
[    43.705] (EE) 3: /nix/store/db8dmfz95lk5pfamvfl9sjg71wnilj7f-malloc-provider-graphene-hardened/lib/libhardened_malloc.so (realloc+0x35d) [0x7bdefced970d]
[    43.705] (EE) 4: /nix/store/aq6widncxzcr8kndsrr42a9c61r39qll-xorg-server-1.20.4/bin/X (XNFreallocarray+0x9) [0x59dc29]
[    43.705] (EE) 5: /nix/store/aq6widncxzcr8kndsrr42a9c61r39qll-xorg-server-1.20.4/bin/X (xf86AllocateEntityPrivateIndex+0x58) [0x475a38]
[    43.705] (EE) 6: /nix/store/xdj3r6d4ci73d65adh57hk6b03qg81lr-libglvnd-1.0.0/lib/libGLdispatch.so.0 (nvidiaAddDrawableHandler+0x4f72da) [0x721f8650e084]
[    43.705] (EE) 
[    43.705] (EE) Segmentation fault at address 0x723e5a4544c0

Steps to reproduce

Use nixpkgs master on a machine with an NVIDIA card.

In the system configuration:

1. Use

services.xserver = { videoDrivers = [ "nvidia" ]; };
hardware.opengl = { enable = true; driSupport = true; };
services.xserver = { enable = true; displayManager.startx.enable = true; };

2. Import <nixpkgs/nixos/modules/profiles/hardened.nix>
3. Keep the default environment.memoryAllocator.provider = "graphene-hardened"

startx and observe a segfault.

Technical details

Please run nix-shell -p nix-info --run "nix-info -m" and paste the
results.

• system: "x86_64-linux"
• host os: Linux 4.19.42-hardened, NixOS, 19.09.git.11851bd (Loris)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.2.2
• channels(root): "nixos-18.09.1922.97e0d53d669"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

---

Maybe environment.memoryAllocator.provider = "libc"; should remain the default in hardened, or maybe graphene-hardened should be marked incompatible with nvidia?

[joachifm]

In an ideal world, crashes caused by legitimate bugs (assuming this is one) would be fixed upstream, but I suppose that's out of the question in this case.

I prefer to keep the default as-is for now and improve docs instead, but I can see how it will be quite annoying for people when things start crashing for no obvious reason ... I'm open to changing the default, obviously, but I'd like to hold off on doing so just a little bit. (To elaborate: the hardened defaults are expected to break things, under the assumption that the admin relaxes settings to suit their specific needs).

[delroth]

I would suggest we repurpose this issue for general brokenness with the Graphene hardened allocator picked by default in the hardened profile.

According to #62238 (comment) it seems like the Graphene hardened allocator can't even run sshd properly. Nix also segfaults / runs out of memory when using that allocator. I think it's probably an OK choice for select applications but definitely not for systemwide use at this point.

[delroth]

(I'm also fine if you want to roll back #62238 temporarily to damage-control before making a decision on this.)

[delroth]

I will slightly amend my statement "can't even run sshd properly" -- the issue seems to be that openssh's seccomp sandbox does not allow mprotect, and Graphene's allocator makes copious use of that syscall (for guard pages). This could be addressed through disabling the seccomp sandbox or applying a custom patch.

[joachifm]

Another possibility is to add service specific overrides to the malloc module to paper over known incompatibilities.

[delroth]

Having to whitelist sshd from these hardening measures would be a pretty sad long term situation though.

[Izorkin]

Service opessh worked with memoryAllocator provider jemalloc and scudo.
With memoryAllocator provider scudo not worked services - nginx unit and dovecot2

[joachifm]

@Izorkin do you think those are fixable in a maintainable way? I'm not too concerned about Nix or proprietary gfx drivers not working, but if things like ssh, httpd &c break, then enabling this by default may be too much.

[Izorkin]

I do not have normal working ideas(

[yotann]

FWIW, I got the same crash with the open-source radeon driver. I solved it by changing environment.memoryAllocator.provider. The problem seems to be in xf86AllocateEntityPrivateIndex:

(EE) Backtrace:
(EE) 0: /nix/store/.../bin/X (OsSigHandler+0x29) [0x59aef9]
(EE) 1: /nix/store/.../lib/libpthread.so.0 (funlockfile+0x50) [0x7f4f176368af]
(EE) 2: /nix/store/.../lib/libc.so.6 (__memmove_avx_unaligned_erms+0x211) [0x7f4f175bebf1]
(EE) 3: /nix/store/.../lib/libhardened_malloc.so (realloc+0x35d) [0x7f4f17cb470d]
(EE) 4: /nix/store/.../bin/X (XNFreallocarray+0x9) [0x59dba9]
(EE) 5: /nix/store/.../bin/X (xf86AllocateEntityPrivateIndex+0x58) [0x4759a8]
(EE) 6: /nix/store/.../lib/xorg/modules/drivers/radeon_drv.so (radeon_platform_probe+0x185) [0x75f31c6fbe85]
(EE) 7: /nix/store/.../bin/X (doPlatformProbe+0x194) [0x49b3b4]
(EE) 8: /nix/store/.../bin/X (probeSingleDevice+0x9a) [0x49b4aa]
(EE) 9: /nix/store/.../bin/X (xf86platformProbeDev+0x18d) [0x49be8d]
(EE) 10: /nix/store/.../bin/X (xf86CallDriverProbe+0xa5) [0x474be5]
(EE) 11: /nix/store/.../bin/X (xf86BusConfig+0x45) [0x475505]
(EE) 12: /nix/store/.../bin/X (InitOutput+0x944) [0x482e04]
(EE) 13: /nix/store/.../bin/X (dix_main+0x18f) [0x4461ff]
(EE) 14: /nix/store/.../lib/libc.so.6 (__libc_start_main+0xee) [0x7f4f1748eb8e]
(EE) 15: /nix/store/.../bin/X (_start+0x2a) [0x43010a]
(EE) 
(EE) Segmentation fault at address 0x75ffd0c12890

[joachifm]

For future reference regarding incompatibilities with hardened_malloc https://github.com/GrapheneOS/hardened_malloc/issues/89#issuecomment-522236117

[ivan]

#66687 was merged, so this should no longer happen.

[joachifm]

@delroth ssh should work with graphene after openssh/openssh-portable@f6906f9