Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixosTests.installer: Port installer and ZFS test to python #78670

Merged
merged 4 commits into from Feb 8, 2020

Conversation

@tfc
Copy link
Contributor

@tfc tfc commented Jan 28, 2020

Motivation for this change

#72828

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

cc @flokli @worldofpeace

@tfc
Copy link
Contributor Author

@tfc tfc commented Jan 28, 2020

@GrahamcOfBorg test installer zfs

@worldofpeace
Copy link
Member

@worldofpeace worldofpeace commented Jan 31, 2020

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

@tfc
Copy link
Contributor Author

@tfc tfc commented Jan 31, 2020

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

Done @worldofpeace

@tfc tfc force-pushed the tfc:port-installer-test branch from 7cc4a1d to ab649e2 Jan 31, 2020
@worldofpeace
Copy link
Member

@worldofpeace worldofpeace commented Jan 31, 2020

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

Done @worldofpeace

Umm I don't see a different commit message for ab649e2?

@tfc tfc force-pushed the tfc:port-installer-test branch from ab649e2 to 380134d Feb 1, 2020
@tfc
Copy link
Contributor Author

@tfc tfc commented Feb 1, 2020

@tfc please reword your commit for the zfs test, it's already ported 40396a7.

Done @worldofpeace

Umm I don't see a different commit message for ab649e2?

Sorry, my fault. Please have a look again @worldofpeace

@@ -566,6 +567,42 @@ def screenshot(self, filename: str) -> None:
if ret.returncode != 0:
raise Exception("Cannot convert screenshot")

def copy_from_host_via_shell(self, source: str, target: str) -> None:

This comment has been minimized.

@flokli

flokli Feb 3, 2020
Contributor

Is there a reason we have this?

Looking at 510ada7, these could probably also just use copy_from_host

This comment has been minimized.

@tfc

tfc Feb 3, 2020
Author Contributor

At first i am using copy_from_host, but with the machine configs that follow later in the script, it doesn't work any longer as the VMs do not seem to support this. For that reason i added the shell variant as an addition.

This comment has been minimized.

@flokli

flokli Feb 3, 2020
Contributor

I assume this is because /tmp/shared from nixos/modules/virtualisation/qemu-vm.nix isn't setup in the installer.

Can we just invoke the mount command manually before running copy_from_host?

This comment has been minimized.

@tfc

tfc Feb 3, 2020
Author Contributor

We could, but i really don't like the implications of this: Currently, tests are agnostic of how this is all implemented and this allows for transparently changing the virt backend etc (apart from the create_machine parameters which are pretty much qemu-specific, but these are also used by only a minority of tests).

As soon as tests begin doing such things, they would become testdriver-implementation specific and i am pretty sure we don't want that. I'd rather have two different functions that communicate which preconditions they assume and let the user choose from them, before users hard-code testdriver-details into their tests.

This comment has been minimized.

@flokli

flokli Feb 3, 2020
Contributor

We had an offline discussion on that. Agreed on not leaking implementation-specific details about how things are shared into the actual test code. Ideally, this should be encapsulated behind the corresponding methods.

def copy_from_host_via_shell(self, source: str, target: str) -> None:
"""Copy a file from the host into the guest by piping it over the
shell into the destination file. Works without host-guest shared folder.
Prefer copy_from_host for large files.

This comment has been minimized.

@flokli

flokli Feb 3, 2020
Contributor

Suggested change
Prefer copy_from_host for large files.
Prefer copy_from_host whenever possible.

This implies it's okay to use it for small files.

In fact, we should use copy_from_host whenever possible, and only use the base64 and shell dance if we have to IMHO.

@tfc tfc force-pushed the tfc:port-installer-test branch 3 times, most recently from 7a47b7c to 11de1ae Feb 3, 2020
@flokli
Copy link
Contributor

@flokli flokli commented Feb 3, 2020

@tfc The zfs tests are still missing from all-tests.nix. Can you add them there?

@tfc tfc force-pushed the tfc:port-installer-test branch from 11de1ae to c2864b8 Feb 3, 2020
@tfc
Copy link
Contributor Author

@tfc tfc commented Feb 3, 2020

@GrahamcOfBorg test zfs

@flokli
Copy link
Contributor

@flokli flokli commented Feb 4, 2020

@GrahamcOfBorg build nixosTests.installer.simpleUefiSystemdBoot

@flokli
Copy link
Contributor

@flokli flokli commented Feb 4, 2020

@worldofpeace
Copy link
Member

@worldofpeace worldofpeace commented Feb 4, 2020

the above test case also fails on master btw https://hydra.nixos.org/build/111477951, but I can confirm it currently fails for an related reasons to that one.

@flokli
Copy link
Contributor

@flokli flokli commented Feb 4, 2020

As the log indicates, this currently fails in the black validator, so it's more broken than before. We should at least fix that.

@flokli
Copy link
Contributor

@flokli flokli commented Feb 4, 2020

Also not sure if the installer tests block channel releases, so we should be extra careful here.

@tfc
Copy link
Contributor Author

@tfc tfc commented Feb 5, 2020

As the log indicates, this currently fails in the black validator, so it's more broken than before. We should at least fix that.

Don't worry, i am on it. It's just that i had no extra time yesterday.

@tfc tfc force-pushed the tfc:port-installer-test branch from c2864b8 to 0495eae Feb 5, 2020
@tfc
Copy link
Contributor Author

@tfc tfc commented Feb 5, 2020

@GrahamcOfBorg test installer.simpleUefiSystemdBoot

@disassembler disassembler modified the milestones: 20.03, 20.09 Feb 5, 2020
@tfc
Copy link
Contributor Author

@tfc tfc commented Feb 5, 2020

So the current state is that the simpleUefiSystemdBoot test sometimes succeeds, sometimes doesn't (on both x86_64 and aarch64). When it fails, it fails like this:

  • nixos-install calls systemd bootctl and that produces:
machine # copying channel...
machine # installing the boot loader...
machine # setting up /etc...
machine # Initializing machine ID from random generator.
machine # Failed to open file system "/dev/block/253:1": No such file or directory
machine # Traceback (most recent call last):
machine #   File "/nix/store/4lzpcmzwdi73ma34vx6hiff80an1r050-systemd-boot-builder.py", line 240, in <module>
machine #     main()
machine #   File "/nix/store/4lzpcmzwdi73ma34vx6hiff80an1r050-systemd-boot-builder.py", line 199, in main
machine #     subprocess.check_call(["/nix/store/c54w4pb6g627ix9d7pyrqfv9qvgjbg6n-systemd-243.4/bin/bootctl", "--path=/boot", "--no-variables", "install"])
machine #   File "/nix/store/4c4c5ig32lqhsgmcncnb72c3a4bwj66n-python3-3.7.6/lib/python3.7/subprocess.py", line 363, in check_call
machine #     raise CalledProcessError(retcode, cmd)
machine # subprocess.CalledProcessError: Command '['/nix/store/c54w4pb6g627ix9d7pyrqfv9qvgjbg6n-systemd-243.4/bin/bootctl', '--path=/boot', '--no-variables', 'install']' returned non-zero exit status 1.
machine: output:
error: command `nixos-install < /dev/null >&2` failed (exit code 1)
cleaning up
killing machine (pid 9)
(0.00 seconds)
builder for '/nix/store/7jad11fkfs1rqql86zx5c84i1dqf7b1m-vm-test-run-installer-simpleUefiSystemdBoot.drv' failed with exit code 1
error: build of '/nix/store/7jad11fkfs1rqql86zx5c84i1dqf7b1m-vm-test-run-installer-simpleUefiSystemdBoot.drv' failed
@tfc tfc force-pushed the tfc:port-installer-test branch from 0495eae to 81172e1 Feb 5, 2020
@tfc
Copy link
Contributor Author

@tfc tfc commented Feb 5, 2020

@GrahamcOfBorg test installer.btrfsSimple installer.luksroot-format1 installer.simpleClone installer.swraid installer.btrfsSubvolDefault installer.luksroot-format2 installer.simpleLabels installer.zfsroot installer.btrfsSubvols installer.lvm installer.simpleProvided installer.encryptedFSWithKeyfile installer.separateBoot installer.simpleUefiGrub installer.grub1 installer.separateBootFat installer.simpleUefiGrubClone installer.luksroot installer.simple installer.simpleUefiSystemdBoot

@tfc tfc force-pushed the tfc:port-installer-test branch from 81172e1 to 30d8d1d Feb 6, 2020
@tfc
Copy link
Contributor Author

@tfc tfc commented Feb 6, 2020

@GrahamcOfBorg test installer-btrfsSimple installer-luksroot-format1 installer-simpleClone installer-swraid installer-btrfsSubvolDefault installer-luksroot-format2 installer-simpleLabels installer-zfsroot installer-btrfsSubvols installer-lvm installer-simpleProvided installer-encryptedFSWithKeyfile installer-separateBoot installer-simpleUefiGrub installer-grub1 installer-separateBootFat installer-simpleUefiGrubClone installer-luksroot installer-simple installer-simpleUefiSystemdBoot

@tfc tfc force-pushed the tfc:port-installer-test branch from 30d8d1d to 942f0fd Feb 7, 2020
@tfc
Copy link
Contributor Author

@tfc tfc commented Feb 7, 2020

@GrahamcOfBorg test installer

@flokli
Copy link
Contributor

@flokli flokli commented Feb 8, 2020

Judging from hydra, simpleUefiSystemdBoot seems to have been flaky in the perl test driver as well, so merging this in.

@flokli flokli merged commit dd5f92f into NixOS:master Feb 8, 2020
14 checks passed
14 checks passed
Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A darwin-tested
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release-combined.nix -A tested
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
tests.installer on aarch64-linux Success
Details
tests.installer on x86_64-linux Success
Details
@worldofpeace
Copy link
Member

@worldofpeace worldofpeace commented Feb 8, 2020

Good for me as well.

dtzWill added a commit to dtzWill/nixpkgs that referenced this pull request Feb 8, 2020
nixosTests.installer: Port installer and ZFS test to python

(cherry picked from commit dd5f92f)
@tfc tfc deleted the tfc:port-installer-test branch Feb 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.