cdituri/feature/hyperv packer builder #58

cdituri · 2020-04-13T00:45:55Z

Overview

This pull request adds hyperv-iso builder support to nix-community/nixbox:

add hyperv packer builder
add hyperv-iso.nix builder
regenerate iso_urls.json, nixos-i686.json, and nixos-x86_64.json

!!! Important: the last commit in the series brings iso_urls.json, nixos-i686.json, and nixos-x86_64.json to current. This is note-worthy because it looks like NixOS upstream is now using a 'latest' .iso release, and we leverage that new approach here as well.

Steps

Run packer build:

packer build -force -var "memory=2048" -var "disk_size=10280" -on-error=ask -only=hyperv-iso .\nixos-x86_64.json

Import resultant vagrant .box image:

vagrant box add --force --name hyperv/nixos-x86_64 .\nixos-19.09-hyperv-x86_64.box

Run vagrant:

vagrant init hyperv/nixos-x86_64
vagrant up --provider=hyperv

Final note: many individuals on Windows have git config --global core.autocrlf true set, myself included. Having autocrlf set caused the following error during packer build, after a fresh clone:

==> hyperv-iso: Installation finished. No error reported.
    hyperv-iso: installation finished!
==> hyperv-iso:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
==> hyperv-iso:                                  Dload  Upload   Total   Spent    Left  Speed
==> hyperv-iso: 100   524  100   524    0     0  32750      0 --:--:-- --:--:-- --:--:-- 34933
==> hyperv-iso: setting up /etc...
==> hyperv-iso: /nix/var/nix/profiles/system/sw/bin/bash: line 2: $'\r': command not found
==> hyperv-iso: /nix/var/nix/profiles/system/sw/bin/bash: line 4: $'\r': command not found
==> hyperv-iso: /nix/var/nix/profiles/system/sw/bin/bash: line 7: $'\r': command not found
==> hyperv-iso: /nix/var/nix/profiles/system/sw/bin/bash: line 8: syntax error near unexpected token `$'do\r''
==> hyperv-iso: '
    hyperv-iso: Start postinstall ...
==> hyperv-iso: Script exited with non-zero exit status: 2.Allowed exit codes are: [0]
==> hyperv-iso: Step "StepProvision" failed
==> hyperv-iso: [c] Clean up and exit, [a] abort without cleanup, or [r] retry step (build may fail even if retry succeeds)?

If you see this error, its because scripts/{install, post-install}.sh need to be LF; and git on Windows has presumably checked them out with CRLF line-endings on clone/checkout.

cdituri · 2020-04-13T00:52:34Z

gen_template.rb

+      builder(
+        type: 'hyperv-iso',
+        generation: 1,
+        # generation: 2, # gen 2 fails uefi boot, missing configuration in builders/hyperv-iso.nix


Packer builds just fine with generation 2 set but the resultant vagrant .box fails at UEFI boot.

Guessing something I missed re: the bootloader and UEFI in scripts/builders/hyperv-iso.nix... was hoping a maintainer or someone from the community may have a better idea here. Gen v1 works no problem however.

cdituri · 2020-04-13T00:58:51Z

gen_template.rb

+          "mkdir -m 0700 .ssh<enter>",
+          "curl $(cat .packer_http)/install_rsa.pub > .ssh/authorized_keys<enter>",
+          "sudo nix-env -f '<nixpkgs>' -iA linuxPackages.hyperv-daemons<enter><wait10>",
+          "sudo $(find /nix/store -executable -iname 'hv_kvp_daemon' | head -n 1)<enter><wait10>",


Not sure if there's a better way to determine the path to hv_kvp_daemon; this daemon needs to be running during live-cd installation in order to pass the dynamically assigned guest-ip back to the hyperv host (in order for packer to continue it's setup over ssh).

Worth note: went this direction, over nixos-rebuild switch and setting virtualisation.hypervGuest.enable in configuration.nix, because the execution time is greatly reduced.

You can get the store path with nix-build --expr "(import <nixpkgs> {}).linuxPackages.hyperv-daemons" for example. But I think it's fine like that. The chances that there are two versions of the same binary is probably fairly low.

The different methods resolve to different paths in the nixstore, seems to have to do with hyperv-daemons vs hyperv-daemons-bin.

Not exactly sure how to properly access the latter however, nix-build --expr "(import <nixpkgs> {}).linuxPackages.hyperv-daemons-bin" does not work. So stuck with the original method using find. Would love to know what the best way is here, but ultimately agree @zimbatm; chances that there are two versions of the same binary is low.

gen_template.rb

zimbatm

👍 on principle. This is going to be hard to maintain as it requires the user to have HyperV installed. But there isn't much we can do, this is the current packer design.

cdituri · 2020-04-23T05:34:19Z

Thanks for the review @zimbatm, much appreciated 👍 crazy at work atm so aiming to circle back here sometime over the weekend to incorporate your feedback.

This is going to be hard to maintain as it requires the user to have HyperV installed. But there isn't much we can do, this is the current packer design.

Totally understand, re: maintenance and packer design. I just want more NixOS! and as long as Windows is the preference at my employer, more than happy to help maintain this native Hyper-V builder 😃

zimbatm · 2020-04-23T09:40:27Z

Sounds good. Let me know when it's ready to be merged :)

@shajra-simspace

The web site put in a newline that breaks the assumption of the regex parse. This commit switches the scraper from regex parsing to XML parsing, which hopefully is more robust. Taken from the submission by @shajra-simspace in: - nix-community#59 - https://github.com/shajra-simspace

NixOS upstream now provides a 'latest' .iso image, so we track latest in iso_urls.json: - https://channels.nixos.org/nixos-20.03/latest-nixos-minimal-x86_64-linux.iso - https://channels.nixos.org/nixos-20.03/latest-nixos-minimal-i686-linux.iso

cdituri · 2020-04-27T00:57:12Z

@zimbatm pull request is ready to merge at the moment. Some things worth nothing below first however:

In the last 2 weeks the NixOS website has changed the way they publish the .iso links, breaking the regex in iso_urls_update.rb. @shajra-simspace left a comment in iso_urls_update.rb web scrape broke #59 switching the implementation from regex parsing to XML parsing. It's a more robust implementation, and I've included an attributed commit 1201013.

@shajra-simspace hope you're ok with the above approach. Any thoughts, or would you like things attributed differently, etc? see the commit link.

NixOS 20.03 has changed users.group and users.users and are depreciating the current form. For nix-community/nixbox I've left things as-is, as it;s the most graceful approach (ie backwards compatible when building <= NixOS 20.03), and warns adequately while building. See NixOS/nikpkgs/pull/63103 and the below.

Click to expand and show warnings generated

==> hyperv-iso: mke2fs 1.45.5 (07-Jan-2020)
    hyperv-iso: Discarding device blocks: done
    hyperv-iso: Creating filesystem with 2631424 4k blocks and 658368 inodes
    hyperv-iso: Filesystem UUID: ed750ed0-be48-4875-9572-ae69f994de08
    hyperv-iso: Superblock backups stored on blocks:
    hyperv-iso:         32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
    hyperv-iso:
    hyperv-iso: Allocating group tables: done
    hyperv-iso: Writing inode tables: done
    hyperv-iso: Creating journal (16384 blocks): done
    hyperv-iso: Writing superblocks and filesystem accounting information: done
    hyperv-iso:
==> hyperv-iso: writing /mnt/etc/nixos/hardware-configuration.nix...
==> hyperv-iso: writing /mnt/etc/nixos/configuration.nix...
    hyperv-iso: building the configuration in /mnt/etc/nixos/configuration.nix...
==> hyperv-iso: trace: warning: In file /nix/var/nix/profiles/per-user/root/channels/nixos/nixos/modules/config/users-groups.nix
==> hyperv-iso: a list is being assigned to the option config.users.users.
==> hyperv-iso: This will soon be an error as type loaOf is deprecated.
==> hyperv-iso: See https://github.com/NixOS/nixpkgs/pull/63103 for more information.
==> hyperv-iso: Do
==> hyperv-iso:   users.users =
==> hyperv-iso:     { root = {...}; vagrant = {...}; }
==> hyperv-iso: instead of
==> hyperv-iso:   users.users =
==> hyperv-iso:     [ { name = "root"; ...} { name = "vagrant"; ...} ]
==> hyperv-iso:
==> hyperv-iso: trace: warning: In file /nix/var/nix/profiles/per-user/root/channels/nixos/nixos/modules/config/users-groups.nix
==> hyperv-iso: a list is being assigned to the option config.users.groups.
==> hyperv-iso: This will soon be an error as type loaOf is deprecated.
==> hyperv-iso: See https://github.com/NixOS/nixpkgs/pull/63103 for more information.
==> hyperv-iso: Do
==> hyperv-iso:   users.groups =
==> hyperv-iso:     { vagrant = {...}; }
==> hyperv-iso: instead of
==> hyperv-iso:   users.groups =
==> hyperv-iso:     [ { name = "vagrant"; ...} ]
==> hyperv-iso:
    hyperv-iso: copying channel...
    hyperv-iso: installing the boot loader...
==> hyperv-iso: setting up /etc...
==> hyperv-iso: updating GRUB 2 menu...
==> hyperv-iso: installing the GRUB 2 boot loader on /dev/sda...
==> hyperv-iso: Installing for i386-pc platform.
==> hyperv-iso: Installation finished. No error reported.
    hyperv-iso: installation finished!

zimbatm · 2020-04-27T12:55:06Z

thanks!

cdituri commented Apr 13, 2020

View reviewed changes

zimbatm reviewed Apr 19, 2020

View reviewed changes

gen_template.rb Outdated Show resolved Hide resolved

zimbatm approved these changes Apr 19, 2020

View reviewed changes

cdituri and others added 5 commits April 26, 2020 19:30

add hyperv packer builder

5251c18

add hyperv-iso.nix builder

c02f2fd

README.md: add packer usage for the hyperv-iso builder

e25f042

cdituri force-pushed the cdituri/feature/hyperv-packer-builder branch from 7b18305 to e25f042 Compare April 27, 2020 00:31

cdituri mentioned this pull request Apr 27, 2020

iso_urls_update.rb web scrape broke #59

Closed

zimbatm merged commit 6bc48b4 into nix-community:master Apr 27, 2020

FranklinYu mentioned this pull request May 12, 2020

Use permanent URL instead of latest #61

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cdituri/feature/hyperv packer builder #58

cdituri/feature/hyperv packer builder #58

cdituri commented Apr 13, 2020 •

edited

cdituri Apr 13, 2020

cdituri Apr 13, 2020 •

edited

cdituri Apr 16, 2020 •

edited

zimbatm Apr 19, 2020

cdituri Apr 27, 2020 •

edited

zimbatm left a comment

cdituri commented Apr 23, 2020 •

edited

zimbatm commented Apr 23, 2020

cdituri commented Apr 27, 2020

zimbatm commented Apr 27, 2020

cdituri/feature/hyperv packer builder #58

cdituri/feature/hyperv packer builder #58

Conversation

cdituri commented Apr 13, 2020 • edited

Overview

Steps

cdituri Apr 13, 2020

Choose a reason for hiding this comment

cdituri Apr 13, 2020 • edited

Choose a reason for hiding this comment

cdituri Apr 16, 2020 • edited

Choose a reason for hiding this comment

zimbatm Apr 19, 2020

Choose a reason for hiding this comment

cdituri Apr 27, 2020 • edited

Choose a reason for hiding this comment

zimbatm left a comment

Choose a reason for hiding this comment

cdituri commented Apr 23, 2020 • edited

zimbatm commented Apr 23, 2020

cdituri commented Apr 27, 2020

zimbatm commented Apr 27, 2020

cdituri commented Apr 13, 2020 •

edited

cdituri Apr 13, 2020 •

edited

cdituri Apr 16, 2020 •

edited

cdituri Apr 27, 2020 •

edited

cdituri commented Apr 23, 2020 •

edited