nixos-rebuild boot fails when xorg-server build script tries to open directories under /var/tmp #19673

peterhoeg · 2016-10-19T05:07:08Z

Issue description

nixos-rebuild boot fails to run due to this:

systemd will create directories under /var/tmp which no user has access to for services with PrivateTmp=true set.
https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/x11/xorg/overrides.nix#L462 replaces the "compiled" directory for the xorg-server package with a link to /var/tmp.
the build script will try to descend into sub directories and fail due to 1.

This is the error:

cannot open `/nix/store/gv307dkp3r5hzj6jmqq2nxlsj2l31i90-xorg-server-1.18.3/share/X11/xkb/compiled/systemd-private-fc254f6d36da46a69c448c429d105be3-firefox@default.service-CgnwMR':
Permission denied at /nix/store/9g4wc31j7a2xp22xpgwr0qssfxahxdzl-builder.pl line 58.

If x/xkb needs to dynamically compile keymaps (I'm guessing that is what that $out/share/X11/xkb/compiled directory is for), shouldn't we instead give it its own directory under
/var/lib/xkb or similar instead?

Cc: @groxxda

Technical details

NixOS: 17.03.git.014efa9M (Gorilla)
Nix: nix-env (Nix) 1.11.4
nixpkgs: 17.03.git.e09f343M

The text was updated successfully, but these errors were encountered:

groxxda · 2016-10-19T15:10:56Z

Maybe relevant for #19629

@peterhoeg can you please confirm my assumptions

this only happens without sandboxing
you have no special settings for the xkb stuff

I'm not convinced this is a xkb specific problem, it may as well be a bug in builder.pl.
At least the path that it tries to access looks very suspicious..

peterhoeg · 2016-10-19T15:35:04Z

@groxxda

it happens with and without useSandbox = true;
it happens with and without setting xkbOptions

groxxda · 2016-10-19T16:36:11Z

@peterhoeg any idea why this failure does not show up on hydra?

dezgeg · 2016-10-19T16:54:36Z

Probably it happens only when systemPackages (or maybe something else that is being buildEnv'd) contains more than one package having a share/X11/xkb directory.

dezgeg · 2016-10-19T20:46:52Z

I wonder if that symlink is needed anymore? Such directory doesn't exist on my Arch Linux installation for instance (edit: nvm, they just override the location to /var/lib/xkb). The code which added that is pretty old: 9d15693

peterhoeg · 2016-10-20T02:58:45Z

Now this is getting odd.

Everything was working for the longest time.
{ nix.useSandbox = true; } all along except for
I started making some changes to the KDE packages, which caused massive rebuilds. I disabled useSandbox in the hope that I could speed up the compilation (that basically the previously compiled stuff wouldn't be thrown away every time)
My machine was now booted with a generation that was built with useSandbox disabled.

And this is where the problems started.

Checking /etc/nix/nix.conf it would show that useSandbox was disabled, but trying to build with (and without) useSandbox enabled would give the error I mentioned.

Stupid me had run nix-collect-garbage -d earlier as the drive was getting VERY full.

So in under to work around it, I did mount -o remount,rw /nix/store and edited /etc/nix/nix.conf to enable useSandbox but keep in mind that in my configuration it was still enabled all along and after a reboot nixos-rebuild works again.

Why would /etc/nix/nix.conf override? Is it because nix-daemon was already started without useSandbox so that takes priority?

dezgeg · 2016-10-20T10:46:16Z

Why would /etc/nix/nix.conf override? Is it because nix-daemon was already started without useSandbox so that takes priority?

Yes, if you modify nix.conf you generally need to restart nix-daemon.

BTW, a safer way to do that is to NOT remount /nix/store rw but instead, make /etc/nix/nix.conf not a symlink but a normal file.

peterhoeg · 2016-10-20T10:57:27Z

Let's say I boot the machine using a generation built with useSandbox = false.

If I then enable useSandbox, I would imagine that the new generation is built in a sandbox as I would otherwise have to do it twice because my first round would not be. Then when rebooting nix.conf would tell me that useSandbox is true but it wasn't true when the generation was built.

joelmo · 2016-10-26T19:59:34Z

I have the same problem. Everything under /var/tmp gets copied to /nix/store/...-xorg-server-1.18.4/share/X11/xkb/compiled.

Maybe it can be fixed by being more specific here: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/x11/xorg/overrides.nix#L462

Say ln -s /var/tmp/systemd-private-* $out...?

peterhoeg · 2016-10-28T09:18:58Z

@joelmo, does it work with nix.useSandbox = true;?

@groxxda, any thoughts on using /var/lib/xkb or similar?

stapelberg · 2016-10-29T14:16:36Z

I ran into the same problem. Setting nix.useSandbox = true fixes the problem for me.

vcunat · 2016-10-29T14:56:00Z

@joelmo: does it really get copied for you? I see only a symlink in the code.

joelmo · 2016-10-30T14:32:15Z

Sorry /nix/store/...-xorg-server-1.18.4/share/X11/xkb/compiled is a symlink. I haven't tried useSandbox but that would probably solve my problem.

dezgeg · 2016-11-07T15:47:05Z

I think we can just do this:

diff --git a/pkgs/servers/x11/xorg/overrides.nix b/pkgs/servers/x11/xorg/overrides.nix
index 930132b..60a7354 100644
--- a/pkgs/servers/x11/xorg/overrides.nix
+++ b/pkgs/servers/x11/xorg/overrides.nix
@@ -509,7 +509,6 @@ in
         '';
         postInstall = ''
           rm -fr $out/share/X11/xkb/compiled
-          ln -s /var/tmp $out/share/X11/xkb/compiled

           cp -rT ${darwinOtherX}/bin $out/bin
           rm -f $out/bin/X

That should just fall back to /tmp/ as per the code:

 48         /*
 49          * If XKM_OUTPUT_DIR specifies a path without a leading slash, it is
 50          * relative to the top-level XKB configuration directory.
 51          * Making the server write to a subdirectory of that directory
 52          * requires some work in the general case (install procedure
 53          * has to create links to /var or somesuch on many machines),
 54          * so we just compile into /usr/tmp for now.
 55          */
 56 #ifndef XKM_OUTPUT_DIR
 57 #define XKM_OUTPUT_DIR  "compiled/"
 58 #endif
 59 
...
 73 
 74 static void
 75 OutputDirectory(char *outdir, size_t size)
 76 {
 77 #ifndef WIN32
 78     /* Can we write an xkm and then open it too? */
 79     if (access(XKM_OUTPUT_DIR, W_OK | X_OK) == 0 &&
 80         (strlen(XKM_OUTPUT_DIR) < size)) {
 81         (void) strcpy(outdir, XKM_OUTPUT_DIR);
 82     }
 83     else
 84 #else
 85     if (strlen(Win32TempDir()) + 1 < size) {
 86         (void) strcpy(outdir, Win32TempDir());
 87         (void) strcat(outdir, "\\");
 88     }
 89     else
 90 #endif
 91     if (strlen("/tmp/") < size) {
 92         (void) strcpy(outdir, "/tmp/");
 93     }
 94 }

Or would an explicit patch to that function be better?

(Arguably though, buildEnv should be fixed to not follow non-store symlinks.)

vcunat · 2016-11-07T15:59:29Z

Sounds OK to me.

sorpaas · 2016-11-13T08:50:27Z

Still the same error after applying patch from @dezgeg. Running 16.09. Any suggestions?

joelmo · 2016-11-13T09:00:05Z

Do someone know what functionality ln -s /var/tmp $out/share/X11/xkb/compiled have? I can remove that and don't notice anything different, running kde5.

sorpaas · 2016-11-13T09:37:58Z

Just fixed the problem (http://git.that.world/nixpkgs.git/commit/?h=infra&id=02d2655fb9e9c8220c761484f1b0c6861af376d3). Turns out there's 2 more ln-s to be removed.

changlinli · 2016-11-16T06:08:08Z

@sorpaas It looks like your fix works for me

luispedro · 2016-11-21T16:52:35Z

This is still failing for me. Does it need to be backported for 16.09?

vcunat · 2016-11-21T16:59:50Z

@luispedro: I believe no fix has been pushed to the official repo yet.

aszlig · 2016-11-29T22:40:35Z

Apart from the oddity that compiled is a symlink to /var/tmp and gets into the output path, I think we should also address this in buildEnv so that it doesn't try to dereference symlinks that are outside of the store.

roxma · 2016-12-03T02:43:51Z

Same problem here, I'm kind of scratching my head now.
Looks like useSandbox = true doesn't work for me.

sheenobu · 2016-12-03T02:47:31Z

@roxma I had to do useSandbox = true, then nixos-rebuild switch (without KDE or KDM enabled) , then check /etc/nix/nix.conf to ensure sandbox was enabled, then systemctl restart nix-daemon, finally you can enable KDE and re-run nixos-rebuild switch.

roxma · 2016-12-03T03:07:20Z

@sheenobu
Thanks, now It's working!

joelmo · 2016-12-03T10:13:28Z

Another way to go around this error is to remove everything in /var/tmp.

ghost · 2016-12-04T15:15:20Z

@vcunat Will it go into nixos-16.09 branch of nixpkgs-channels? Thanks.

vcunat · 2016-12-04T15:26:17Z

@gnidorah, I planned to wait a few days so it gets more testing.

- Fixes #19673; it caused problems in combination with buildEnv. - As noted, X falls back to /tmp: #19673 (comment) - Removing the directory is still required, as X would attempt to write into it if allowed - and probably succeed in case the user set nix.readOnlyStore = false; (X runs as root). - Archeology link: 9d15693. (cherry picked from commit 33abc70)

vcunat · 2016-12-18T21:27:09Z

Picked to 16.09. It's a bit later than I anticipated, and moreover channels are stalled due to Hydra's current setup.

- Fixes NixOS#19673; it caused problems in combination with buildEnv. - As noted, X falls back to /tmp: NixOS#19673 (comment) - Removing the directory is still required, as X would attempt to write into it if allowed - and probably succeed in case the user set nix.readOnlyStore = false; (X runs as root). - Archeology link: 9d15693. (cherry picked from commit 33abc70)

peterhoeg changed the title ~~nixos-rebuild boot fails when xorg-server build script is trying to open directories under /var/tmp~~ nixos-rebuild boot fails when xorg-server build script tries to open directories under /var/tmp Oct 19, 2016

NeQuissimus added the 0.kind: bug Something is broken label Oct 19, 2016

dezgeg mentioned this issue Nov 7, 2016

xwayland: symlink causes nixos-rebuild to fail #20235

Closed

sheenobu mentioned this issue Dec 3, 2016

Fails to install nixos, cannot open `/nix/store/sm50zmhh42piya38afcq1cqq19mmdr10-xorg-server-1.18.4/share/X11/xkb/compiled/kdecache-root': Permission denied at /nix/store/9g4wc31j7a2xp22xpgwr0qssfxahxdzl-builder.pl line 58 #20878

Closed

vcunat self-assigned this Dec 3, 2016

vcunat closed this as completed in 33abc70 Dec 4, 2016

vcunat mentioned this issue Dec 18, 2016

Nix produced store path with 700 permissions (should be 755) NixOS/nix#1155

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos-rebuild boot fails when xorg-server build script tries to open directories under /var/tmp #19673

nixos-rebuild boot fails when xorg-server build script tries to open directories under /var/tmp #19673

peterhoeg commented Oct 19, 2016

groxxda commented Oct 19, 2016

peterhoeg commented Oct 19, 2016

groxxda commented Oct 19, 2016

dezgeg commented Oct 19, 2016

dezgeg commented Oct 19, 2016 •

edited

Loading

peterhoeg commented Oct 20, 2016

dezgeg commented Oct 20, 2016

peterhoeg commented Oct 20, 2016

joelmo commented Oct 26, 2016

peterhoeg commented Oct 28, 2016

stapelberg commented Oct 29, 2016

vcunat commented Oct 29, 2016

joelmo commented Oct 30, 2016

dezgeg commented Nov 7, 2016

vcunat commented Nov 7, 2016

sorpaas commented Nov 13, 2016

joelmo commented Nov 13, 2016

sorpaas commented Nov 13, 2016

changlinli commented Nov 16, 2016

luispedro commented Nov 21, 2016

vcunat commented Nov 21, 2016

aszlig commented Nov 29, 2016

roxma commented Dec 3, 2016 •

edited

Loading

sheenobu commented Dec 3, 2016

roxma commented Dec 3, 2016

joelmo commented Dec 3, 2016

ghost commented Dec 4, 2016 •

edited by ghost

Loading

vcunat commented Dec 4, 2016

vcunat commented Dec 18, 2016

nixos-rebuild boot fails when xorg-server build script tries to open directories under /var/tmp #19673

nixos-rebuild boot fails when xorg-server build script tries to open directories under /var/tmp #19673

Comments

peterhoeg commented Oct 19, 2016

Issue description

Technical details

groxxda commented Oct 19, 2016

peterhoeg commented Oct 19, 2016

groxxda commented Oct 19, 2016

dezgeg commented Oct 19, 2016

dezgeg commented Oct 19, 2016 • edited Loading

peterhoeg commented Oct 20, 2016

dezgeg commented Oct 20, 2016

peterhoeg commented Oct 20, 2016

joelmo commented Oct 26, 2016

peterhoeg commented Oct 28, 2016

stapelberg commented Oct 29, 2016

vcunat commented Oct 29, 2016

joelmo commented Oct 30, 2016

dezgeg commented Nov 7, 2016

vcunat commented Nov 7, 2016

sorpaas commented Nov 13, 2016

joelmo commented Nov 13, 2016

sorpaas commented Nov 13, 2016

changlinli commented Nov 16, 2016

luispedro commented Nov 21, 2016

vcunat commented Nov 21, 2016

aszlig commented Nov 29, 2016

roxma commented Dec 3, 2016 • edited Loading

sheenobu commented Dec 3, 2016

roxma commented Dec 3, 2016

joelmo commented Dec 3, 2016

ghost commented Dec 4, 2016 • edited by ghost Loading

vcunat commented Dec 4, 2016

vcunat commented Dec 18, 2016

dezgeg commented Oct 19, 2016 •

edited

Loading

roxma commented Dec 3, 2016 •

edited

Loading

ghost commented Dec 4, 2016 •

edited by ghost

Loading