New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to Nix 2.0 #34636

Merged
merged 38 commits into from Mar 3, 2018

Conversation

@edolstra
Member

edolstra commented Feb 5, 2018

Hopefully we can switch to Nix 2.0 for NixOS 18.03. Nix 2.0 should be released in the next few weeks. This PR makes 2.0 the default (by temporarily pointing nix to nixUnstable) and updates nixos-install to work with it. It also adds a command nixos-enter that factors out the functionality of nixos-install --chroot.

edolstra added some commits Feb 5, 2018

Add a "nixos-enter" command
This factors out the functionality in nixos-install for running a
command inside a NixOS installation (nixos-install --chroot).
nixos-install: Make compatible with Nix 2.0
The use of Nix 2.0 significantly simplifies the installer, since we
can just pass a different store URI (--store /mnt) - it's no longer
needed to set up a chroot environment for the build, and to bootstrap
Nix into the chroot.

Also, commands that need to run in the installation (namely boot
loader installation and setting a root password) are now executed
using nixos-enter.

This also removes the need for nixos-prepare-root since any required
initialisation is done by Nix or by the activation script.

edolstra added some commits Feb 5, 2018

modprobe activation: Order after specialfs
It requires the existence of /proc.
nixos-enter: Don't mount special filesystems
The activation script already does this.
Move creation of /root to the activation script
...so it appears in a new installation before rebooting the system.
ISO images: Initialize the Nix database with correct NAR hashes/sizes
The boot test now runs "nix verify" to ensure that all hashes are
correct.
nixos-install: Don't require root
E.g.

  nixos-install --root /tmp/mnt/ --no-bootloader --no-root-passwd

now works for non-root users.
Remove creation of /dev/{fd,stdin,stdout,stderr}
This is already provided by devtmpfs.
nixos-enter: Don't require root
Of course, you'll get a bunch of warnings from the activation script:

  $ nixos-enter --root /tmp/mnt/
  setting up /etc...
  mount: /dev: permission denied.
  mount: /dev/pts: permission denied.
  mount: /dev/shm: permission denied.
  mount: /sys: permission denied.
  /nix/var/nix/profiles/system/activate: line 74: /proc/sys/kernel/modprobe: Permission denied
  chown: changing ownership of '/run/wrappers/wrappers.0pKlU8JsvV/dbus-daemon-launch-helper': Invalid argument
  NOTE: Under Linux, effective file capabilities must either be empty, or
        exactly match the union of selected permitted and inheritable bits.
  Failed to set capabilities on file `/run/wrappers/wrappers.0pKlU8JsvV/ping' (Operation not permitted)
  chown: changing ownership of '/run/wrappers/wrappers.0pKlU8JsvV/unix_chkpwd': Invalid argument

  [root@nixos:/]#
nixos-install: Accept --substituters
This is useful in tests where we don't have network access. Passing
--substituters "" prevents wasting time by checking cache.nixos.org.
make-disk-image.nix: Use nixos-install again
Since nixos-install doesn't require any special privileges anymore,
this Just Works. No more need for fakeroot / nixos-prepare-root.
nixos-prepare-root: Remove
This is no longer needed thanks to Nix 2.0 magic.
Doh
@dezgeg

This comment has been minimized.

Show comment
Hide comment
@dezgeg

dezgeg Feb 7, 2018

Contributor

Do these installer changes also fix #27307 ?

Contributor

dezgeg commented Feb 7, 2018

Do these installer changes also fix #27307 ?

# Run the activation script. Set $LOCALE_ARCHIVE to supress some Perl locale warnings.
LOCALE_ARCHIVE=$system/sw/lib/locale/locale-archive chroot "$mountPoint" "$system/activate" >&2 || true

This comment has been minimized.

@Mic92

Mic92 Feb 9, 2018

Contributor

Bind mounting /etc/resolv.conf would be nice to have, if this tool is supposed to be a general-purpose tool.

@Mic92

Mic92 Feb 9, 2018

Contributor

Bind mounting /etc/resolv.conf would be nice to have, if this tool is supposed to be a general-purpose tool.

This comment has been minimized.

@edolstra

edolstra Feb 28, 2018

Member

Yeah that would be nice.

@edolstra

edolstra Feb 28, 2018

Member

Yeah that would be nice.

@dezgeg dezgeg referenced this pull request Feb 19, 2018

Open

use overlayfs by default for netboot and iso #35188

4 of 8 tasks complete
@dtzWill

This comment has been minimized.

Show comment
Hide comment
@dtzWill

dtzWill Feb 22, 2018

Contributor

Looks like some copying oversight here:

  • sha256 hopefully didn't change
  • perl-bindings probably want to be using nix1 not nixStable, although maybe not.

Looks like some copying oversight here:

  • sha256 hopefully didn't change
  • perl-bindings probably want to be using nix1 not nixStable, although maybe not.
Doh
Show outdated Hide outdated pkgs/tools/package-management/nix/default.nix
@@ -155,7 +155,7 @@ in rec {
name = "nix-1.11.16";
src = fetchurl {
url = "http://nixos.org/releases/nix/${name}/${name}.tar.xz";
sha256 = "7024d327314bf92c1d3e6cccd944929828a44b24093954036bfb0115a92f5a14";
sha256 = "0ca5782fc37d62238d13a620a7b4bff6a200bab1bd63003709249a776162357c";
};
}) // { perl-bindings = nixStable; };

This comment has been minimized.

@dtzWill

dtzWill Feb 22, 2018

Contributor

Not to nag, but just checking: is this intended to use perl bindings from nixStable (nix 2)?

@dtzWill

dtzWill Feb 22, 2018

Contributor

Not to nag, but just checking: is this intended to use perl bindings from nixStable (nix 2)?

edolstra added some commits Feb 22, 2018

Doh
@edolstra

This comment has been minimized.

Show comment
Hide comment
@edolstra

edolstra Feb 22, 2018

Member

@dtzWill Thanks, fixed.

Member

edolstra commented Feb 22, 2018

@dtzWill Thanks, fixed.

@edolstra

This comment has been minimized.

Show comment
Hide comment
@edolstra

edolstra Feb 22, 2018

Member

@dezgeg Yes, the entire build is done in the target filesystem.

Member

edolstra commented Feb 22, 2018

@dezgeg Yes, the entire build is done in the target filesystem.

@edolstra

This comment has been minimized.

Show comment
Hide comment
Merge pull request #35402 from shlevy/closure-info-total-size
closureInfo: Report the total closure size.
url = "http://nixos.org/releases/nix/${name}/${name}.tar.xz";
sha256 = "7024d327314bf92c1d3e6cccd944929828a44b24093954036bfb0115a92f5a14";
};
}) // { perl-bindings = perl-bindings { nix = nixStable; }; };

This comment has been minimized.

@kballard

kballard Feb 23, 2018

I thought Nix 2.0 removed the dependency on Perl?

@kballard

kballard Feb 23, 2018

I thought Nix 2.0 removed the dependency on Perl?

This comment has been minimized.

@dtzWill

dtzWill Feb 23, 2018

Contributor

It does, the perl bindings are for using Nix from perl. Previously nix utilities and such were themselves implemented in perl, which is no longer the case.

@dtzWill

dtzWill Feb 23, 2018

Contributor

It does, the perl bindings are for using Nix from perl. Previously nix utilities and such were themselves implemented in perl, which is no longer the case.

@copumpkin

This comment has been minimized.

Show comment
Hide comment
@copumpkin

copumpkin Feb 24, 2018

Member

Also relevant for 17.09: #35468

Member

copumpkin commented Feb 24, 2018

Also relevant for 17.09: #35468

edolstra added some commits Feb 27, 2018

Fix installing the Nixpkgs channel on the installation media
And test that it got installed correctly.
Add jq to the installation media
This is required by closureInfo.
nixos-install: Create /root/.nix-defexpr
This was previously done by nixos-prepare-root.
Create /home with the right permissions
Without this, it will be created with 700 permissions.
# Copy the NixOS/Nixpkgs sources to the target as the initial contents
# of the NixOS channel.
if [[ -z $noChannelCopy ]]; then
if [[ -z $channelPath ]]; then

This comment has been minimized.

@copumpkin

copumpkin Feb 27, 2018

Member

Ooh, does this make the channel optional? I was just thinking of adding that feature to make-disk-image.nix, since it's quite annoying in many situations.

@copumpkin

copumpkin Feb 27, 2018

Member

Ooh, does this make the channel optional? I was just thinking of adding that feature to make-disk-image.nix, since it's quite annoying in many situations.

This comment has been minimized.

@edolstra

edolstra Feb 28, 2018

Member

Well, it was already optional since we had the --no-channel-copy flag.

@edolstra

edolstra Feb 28, 2018

Member

Well, it was already optional since we had the --no-channel-copy flag.

This comment has been minimized.

@copumpkin

copumpkin Feb 28, 2018

Member

Oh I guess I just haven't exposed that in the disk image machinery. I'll probably take a look after this is merged.

@copumpkin

copumpkin Feb 28, 2018

Member

Oh I guess I just haven't exposed that in the disk image machinery. I'll probably take a look after this is merged.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Feb 27, 2018

Member

Looks like the hydra job (and of borg) have passed it?!?!?!

Member

Ericson2314 commented Feb 27, 2018

Looks like the hydra job (and of borg) have passed it?!?!?!

@edolstra

This comment has been minimized.

Show comment
Hide comment
@edolstra

edolstra Feb 28, 2018

Member

@vcunat Jobset is all green now so this should be good to merge.

Member

edolstra commented Feb 28, 2018

@vcunat Jobset is all green now so this should be good to merge.

@edolstra edolstra changed the title from [WIP] Switch to Nix 2.0 to Switch to Nix 2.0 Mar 2, 2018

@shlevy

shlevy approved these changes Mar 2, 2018

🚢

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Mar 2, 2018

Member

....did @vcunat want to have a final say on this, or @edolstra did you just ping him as one of the release managers?

Member

Ericson2314 commented Mar 2, 2018

....did @vcunat want to have a final say on this, or @edolstra did you just ping him as one of the release managers?

@edolstra

This comment has been minimized.

Show comment
Hide comment
@edolstra

edolstra Mar 2, 2018

Member

Yeah, it should be up to the release managers.

Member

edolstra commented Mar 2, 2018

Yeah, it should be up to the release managers.

@vcunat

This comment has been minimized.

Show comment
Hide comment
@vcunat

vcunat Mar 2, 2018

Member

If you feel it's ready, I believe you can go along. I haven't found time to really read the changes yet.

Member

vcunat commented Mar 2, 2018

If you feel it's ready, I believe you can go along. I haven't found time to really read the changes yet.

@copumpkin

This comment has been minimized.

Show comment
Hide comment
@copumpkin

copumpkin Mar 2, 2018

Member

I haven't had a chance to test the image building changes yet, but don't let me hold you up. If something goes wrong, I'll fix it or talk to you 😄

Member

copumpkin commented Mar 2, 2018

I haven't had a chance to test the image building changes yet, but don't let me hold you up. If something goes wrong, I'll fix it or talk to you 😄

@7c6f434c

This comment has been minimized.

Show comment
Hide comment
@7c6f434c

7c6f434c Mar 2, 2018

Member

Now that the builds are performed in the target store, should it be documented that to have /tmp for the build also in the target filesystem, the user should set TMPDIR? (Or do I misunderstand the situation?)

Member

7c6f434c commented Mar 2, 2018

Now that the builds are performed in the target store, should it be documented that to have /tmp for the build also in the target filesystem, the user should set TMPDIR? (Or do I misunderstand the situation?)

@@ -116,14 +116,22 @@ in rec {
nix = nixStable;
nixStable = (common rec {
nix1 = (common rec {
name = "nix-1.11.16";
src = fetchurl {
url = "http://nixos.org/releases/nix/${name}/${name}.tar.xz";
sha256 = "0ca5782fc37d62238d13a620a7b4bff6a200bab1bd63003709249a776162357c";
};
}) // { perl-bindings = nixStable; };

This comment has been minimized.

@vcunat

vcunat Mar 3, 2018

Member

@edolstra: is it intentional that perl-bindings for nix1 come from are nix2?

@vcunat

vcunat Mar 3, 2018

Member

@edolstra: is it intentional that perl-bindings for nix1 come from are nix2?

This comment has been minimized.

@vcunat

vcunat Mar 3, 2018

Member

Well, I see it's wrong, as nixStable itself has no perl stuff inside, but I still don't know surely if it should be nixStable.perl-bindings or perl-bindings { nix = nix1; }.

@vcunat

vcunat Mar 3, 2018

Member

Well, I see it's wrong, as nixStable itself has no perl stuff inside, but I still don't know surely if it should be nixStable.perl-bindings or perl-bindings { nix = nix1; }.

This comment has been minimized.

@Ericson2314

Ericson2314 Mar 3, 2018

Member

Yeah that looks like a mistake. That isn't even Nix 2's perl bindings.

@Ericson2314

Ericson2314 Mar 3, 2018

Member

Yeah that looks like a mistake. That isn't even Nix 2's perl bindings.

vcunat added some commits Mar 3, 2018

@vcunat

vcunat approved these changes Mar 3, 2018

The binding issue seems a nitpick, really. It's only referenced by hydra and that seems unlikely to be used with nix1 anymore...

I can't see any other problem.

@vcunat vcunat closed this Mar 3, 2018

@vcunat vcunat deleted the nix-2.0 branch Mar 3, 2018

@Mic92 Mic92 merged commit 729d72f into master Mar 3, 2018

@vcunat

This comment has been minimized.

Show comment
Hide comment
@vcunat

vcunat Mar 3, 2018

Member

^^ hehe, GitHub confusing people again. I have reported such case to them once or twice during the past year or two, but apparently it still happens sometimes.

Member

vcunat commented Mar 3, 2018

^^ hehe, GitHub confusing people again. I have reported such case to them once or twice during the past year or two, but apparently it still happens sometimes.

@vcunat

This comment has been minimized.

Show comment
Hide comment
@vcunat

vcunat Mar 3, 2018

Member

Possibly a problem: #36268

Member

vcunat commented Mar 3, 2018

Possibly a problem: #36268

@dezgeg

This comment has been minimized.

Show comment
Hide comment
@dezgeg

dezgeg Apr 14, 2018

Contributor

I think these changes caused #38477. Previously, a mount --rbind /sys was done, but no longer, so I bet efivarfs is not getting mounted inside the nixos-enter chroot anymore.

Contributor

dezgeg commented Apr 14, 2018

I think these changes caused #38477. Previously, a mount --rbind /sys was done, but no longer, so I bet efivarfs is not getting mounted inside the nixos-enter chroot anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment