Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixosTests.ec2: Port tests that depend on common/ec2.nix #79696

Merged
merged 3 commits into from Aug 23, 2020

Conversation

@tfc
Copy link
Contributor

@tfc tfc commented Feb 10, 2020

Motivation for this change

#72828

This is a bundle of ports because it is not possible to port only a portion of the tests that depend on common/ec2.nix unfortunately.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@tfc tfc requested review from worldofpeace and flokli Feb 10, 2020
@tfc
Copy link
Contributor Author

@tfc tfc commented Feb 10, 2020

@GrahamcOfBorg test openstack-image ec2

@veprbl
Copy link
Member

@veprbl veprbl commented Feb 11, 2020

@GrahamcOfBorg test ec2-nixops
@GrahamcOfBorg test openstack-image-userdata
@GrahamcOfBorg test openstack-image-metadata

@FRidh FRidh added this to the 20.03 milestone Feb 13, 2020
$startCommand .= " -drive file=$diskImage,if=virtio,werror=report";
$startCommand .= " \$QEMU_OPTS";
image_dir = os.path.join(
os.environ.get("TMPDIR", tempfile.gettempdir()), "tmp", "vm-state-machine"

This comment has been minimized.

@flokli

flokli Feb 13, 2020
Contributor

do you know why we create new temporary folders here?

I'd expect it to be due workaround leftover files when invoking nix-build nixos/tests/ec2.nix, but I'm not sure… Did you check the commit history?

If that's the case, some comment might help.

This comment has been minimized.

@nlewo

nlewo Apr 4, 2020
Member

I didn't check the history, but it could be to reproduce the behavior of other tests: if you start the test interactively, the VM state files are created in the /tmp/vm-<machine-name>/state directory.

# again when it deletes link-local addresses.) Ideally we'd
# turn off the DHCP server, but qemu does not have an option
# to do that.
start_command = (

This comment has been minimized.

@flokli

flokli Feb 13, 2020
Contributor

I don't really understand why we invoke qemu here manually - shouldn't that be job of the testing harness and abstracted away entirely? If it's just to pass in the metadata, maybe add a knob in the harness, and wrap the creation of the metadata inside a nix derivation, instead of the test driver script?

This comment has been minimized.

@tfc

tfc Feb 13, 2020
Author Contributor

I don't know much about the amazon ami image stuff in nixpkgs. But my guess is that the image that config.system.build.amazonImage create is a format that qemu doesn't directly consume, so the test creates a .qcow1 file.

And then, that together with the DHCP foo and webservice is a bit too much for the normal machine abstraction.

This comment has been minimized.

@nlewo

nlewo Apr 4, 2020
Member

I think a new qemu image is created because the provided image is readonly since it is in the store. This new qemu image uses "copy on write" from provided image.

Then, a comment mentions an issue with the qemu embeded dhcp server. To make it working, the network configuration of the machine needs to be explicitly specified.

I think this use case is really different from other NIxOS tests because we use a prebuilt image (no shared store...), and we need to do some tricks on the network configuration of the machine. This could explain why this test is a bit different!

@tfc
Copy link
Contributor Author

@tfc tfc commented Feb 13, 2020

@GrahamcOfBorg test ec2-config

@flokli
Copy link
Contributor

@flokli flokli commented Apr 3, 2020

@nlewo would you be able to take a look at this?

@nlewo
nlewo approved these changes Apr 4, 2020
Copy link
Member

@nlewo nlewo left a comment

Thanks for this PR!

$startCommand .= " -drive file=$diskImage,if=virtio,werror=report";
$startCommand .= " \$QEMU_OPTS";
image_dir = os.path.join(
os.environ.get("TMPDIR", tempfile.gettempdir()), "tmp", "vm-state-machine"

This comment has been minimized.

@nlewo

nlewo Apr 4, 2020
Member

I didn't check the history, but it could be to reproduce the behavior of other tests: if you start the test interactively, the VM state files are created in the /tmp/vm-<machine-name>/state directory.

# again when it deletes link-local addresses.) Ideally we'd
# turn off the DHCP server, but qemu does not have an option
# to do that.
start_command = (

This comment has been minimized.

@nlewo

nlewo Apr 4, 2020
Member

I think a new qemu image is created because the provided image is readonly since it is in the store. This new qemu image uses "copy on write" from provided image.

Then, a comment mentions an issue with the qemu embeded dhcp server. To make it working, the network configuration of the machine needs to be explicitly specified.

I think this use case is really different from other NIxOS tests because we use a prebuilt image (no shared store...), and we need to do some tricks on the network configuration of the machine. This could explain why this test is a bit different!

@flokli
Copy link
Contributor

@flokli flokli commented Apr 5, 2020

I rebased on master and couldn't get it to build:

machine # [   79.411051] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: building '/nix/store/7mrpqb0z742m7y61npm0a5s33gyvnjkx-stage-2-init.sh.drv'...
machine # [   89.456673] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: building '/nix/store/7x9q5qdd27aljvvrnrvabb6h6b0sh082-01-fix-ftbfs-with-glibc-2.28.patch.drv'...
machine # [   99.476079] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: warning: unable to download 'http://tarballs.nixos.org/sha256/13874095e1fa9faab45d95a348cf57adb118b3204c5e3881df8e265a976d958a': Couldn't resolve host name (6); retrying in 293 ms
machine # [  109.785618] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: warning: unable to download 'http://tarballs.nixos.org/sha256/13874095e1fa9faab45d95a348cf57adb118b3204c5e3881df8e265a976d958a': Couldn't resolve host name (6); retrying in 694 ms
machine # [  120.494934] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: warning: unable to download 'http://tarballs.nixos.org/sha256/13874095e1fa9faab45d95a348cf57adb118b3204c5e3881df8e265a976d958a': Couldn't resolve host name (6); retrying in 1196 ms
machine # [  122.461717] systemd-timesyncd[554]: Timed out waiting for reply from 169.254.169.123:123 (169.254.169.123).
machine # [  131.701255] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: warning: unable to download 'http://tarballs.nixos.org/sha256/13874095e1fa9faab45d95a348cf57adb118b3204c5e3881df8e265a976d958a': Couldn't resolve host name (6); retrying in 2174 ms
machine # [  153.902935] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: warning: unable to download 'https://sources.debian.org/data/main/m/m4/1.4.18-2/debian/patches/01-fix-ftbfs-with-glibc-2.28.patch': Couldn't resolve host name (6); retrying in 307 ms
machine # [  164.224312] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: warning: unable to download 'https://sources.debian.org/data/main/m/m4/1.4.18-2/debian/patches/01-fix-ftbfs-with-glibc-2.28.patch': Couldn't resolve host name (6); retrying in 548 ms
machine # [  174.787296] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: warning: unable to download 'https://sources.debian.org/data/main/m/m4/1.4.18-2/debian/patches/01-fix-ftbfs-with-glibc-2.28.patch': Couldn't resolve host name (6); retrying in 1395 ms
machine # [  186.195146] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: warning: unable to download 'https://sources.debian.org/data/main/m/m4/1.4.18-2/debian/patches/01-fix-ftbfs-with-glibc-2.28.patch': Couldn't resolve host name (6); retrying in 2183 ms
machine # [  198.392717] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: error: unable to download 'https://sources.debian.org/data/main/m/m4/1.4.18-2/debian/patches/01-fix-ftbfs-with-glibc-2.28.patch': Couldn't resolve host name (6)
machine # [  198.430122] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: builder for '/nix/store/7x9q5qdd27aljvvrnrvabb6h6b0sh082-01-fix-ftbfs-with-glibc-2.28.patch.drv' failed with exit code 1
machine # [  198.440410] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: building '/nix/store/wfa1zpp6x755k4m55mf1cixvzmk7rvq1-CVE-2019-17498.patch.drv'...
machine # [  198.444018] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/0bg63y0aid6w7nik0v6y7jq6asjlnlcv-gnum4-1.4.18.drv': 1 dependencies couldn't be built
machine # [  198.448073] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/6rw7xp1czj8himpmw0b7yl7p7sh2q49v-bison-3.5.1.drv': 1 dependencies couldn't be built
machine # [  198.453037] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/9n3nqn6f1rh58azk7znwjmiw18rqdfnm-gmp-6.2.0.drv': 1 dependencies couldn't be built
machine # [  198.455685] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/vlpsfz9ndpwgr54hn07b7zr2k954fdfm-binutils-2.31.1.drv': 1 dependencies couldn't be built
machine # [  198.463289] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/lif8p6cb8nddka3xnz56l5j9c7w1kzag-gcc-9.3.0.drv': 1 dependencies couldn't be built
machine # [  198.469330] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/icmid690z6da98x2nlcbq6d4si26aa99-binutils-wrapper-2.31.1.drv': 1 dependencies couldn't be built
machine # [  198.474297] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/phwsxivb7jrfv5zzzvxh3xanp7wb798a-binutils-wrapper-2.31.1.drv': 1 dependencies couldn't be built
machine # [  198.480739] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/x0ycqp4j5xn3vwnwqg1jm5nh1h0sx02q-bootstrap-stage4-gcc-wrapper-9.3.0.drv': 1 dependencies couldn't be built
machine # [  198.485245] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/8jkpi4d4zgh9qn8fsw0qy35hqpfimbk1-bootstrap-stage2-gcc-wrapper-.drv': 1 dependencies couldn't be built
machine # [  198.495814] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/7v38861iv8sj4l6rl10ba89skbqjviyb-bootstrap-stage3-gcc-wrapper-.drv': 1 dependencies couldn't be built
machine # [  198.499311] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/rl4b0y8jdr404zivk7q75nwlvzkq6d86-bootstrap-stage4-stdenv-linux.drv': 1 dependencies couldn't be built
machine # [  198.507270] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/piqpddi8mjlzpmmacr7nd9immivxpa74-bootstrap-stage3-stdenv-linux.drv': 1 dependencies couldn't be built
machine # [  198.515625] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/y6rzdhawd24bw0s4igik67y2qicdp516-coreutils-8.31.drv': 1 dependencies couldn't be built
machine # [  198.526320] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/m5h6wa96ydpf515q0vbaivz462x5350p-openssl-1.1.1d.drv': 1 dependencies couldn't be built
machine # [  198.531765] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/c9kg3qq0fcawkzwgsg8ax99nagykqrsk-zlib-1.2.11.drv': 1 dependencies couldn't be built
machine # [  198.542188] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: building '/nix/store/8h09inxs1j8x94sj0lrxz0shdsakvk75-builder.pl.drv'...
machine # [  198.546232] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/8ldy3r1vqs4hl0jgw4baqh0c88dbawkm-curl-7.68.0.drv': 1 dependencies couldn't be built
machine # [  198.551816] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/bxizrb7bv4hvd9fvdrmx84yz9nfsclk1-klibc-2.0.7.tar.xz.drv': 1 dependencies couldn't be built
machine # [  198.559722] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/dqvxsiqc66brf4ixmxvsw8hjp3jzf2hw-linux-5.5.tar.xz.drv': 1 dependencies couldn't be built
machine # [  198.562130] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/bma8qrw3x2c60zjwxinz15556i83rh7k-klibc-2.0.7.drv': 1 dependencies couldn't be built
machine # [  198.569330] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/cz6ydkhmxh1ih4mnwps474v2pl2b3nrg-extra-utils.drv': 1 dependencies couldn't be built
machine # [  198.571844] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/sp08m91sfrapzbgcjiwk3j4p57l17mk0-stage-1-init.sh.drv': 1 dependencies couldn't be built
machine # [  198.577257] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/01lm8pld4ms925qhjw1fp3akcbcflbir-initrd-linux-5.4.30.drv': 1 dependencies couldn't be built
machine # [  198.584392] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: cannot build derivation '/nix/store/1xwr3npi2mlyj7k8npynr7ds1ss3w1x1-nixos-system-ec2-test-vm-20.09pre-git.drv': 1 dependencies couldn't be built
machine # [  198.587489] aij7kdhix9pka4p2kz1gk77jvzqaka88-unit-script-amazon-init-start[808]: error: build of '/nix/store/1xwr3npi2mlyj7k8npynr7ds1ss3w1x1-nixos-system-ec2-test-vm-20.09pre-git.drv' failed
machine # [  198.590320] systemd[1]: amazon-init.service: Main process exited, code=exited, status=1/FAILURE
machine # [  198.592602] systemd[1]: amazon-init.service: Failed with result 'exit-code'.
machine # [  198.594124] systemd[1]: Failed to start Reconfigure the system from EC2 userdata on startup.
machine # [  198.595661] systemd[1]: Startup finished in 2.708s (kernel) + 3min 15.873s (userspace) = 3min 18.581s.
machine # [  198.597411] systemd[1]: amazon-init.service: Consumed 5.683s CPU time, no IP traffic.
error: unit "amazon-init.service" reached state "failed"


@flokli
Copy link
Contributor

@flokli flokli commented Apr 6, 2020

@tfc can you rebase this and give it another look?

tfc and others added 3 commits Feb 2, 2020
This test wants to download things from the internet while building the
system. It can probably be fixed by ensuring these paths are present in
the initial nix-store.
@flokli
Copy link
Contributor

@flokli flokli commented Aug 23, 2020

It seems the merge conflicts resolved themselves (or GitHub doesn't show a git push that happened).

Anyhow - ec2-nixops succeeds, but ec2-config still fails, because amazon-init wants to download things from the internet (which doesn't seem to be in the closure of the building VM), and the VM test runs without network access.

I assume more things need to be added to the initial closure to make it succeed, but the "porting parts" of this test seem to be fine.

I'll mark ec2-config as broken, merge this in and will open a followup issue to fix ec2-config.

@flokli flokli force-pushed the tfc:port-ec2-test branch from ea31c2c to 97a32bc Aug 23, 2020
@flokli flokli merged commit 4460cda into NixOS:master Aug 23, 2020
@flokli
Copy link
Contributor

@flokli flokli commented Aug 23, 2020

I opened #96069 to unbreak nixosTests.ec2-config.

vcunat added a commit that referenced this pull request Oct 3, 2020
The problem was introduced by commit 97a32bc (within PR #79696).
nixos/tests/common/ec2.nix:6:17 called with unexpected argument 'meta'
siraben added a commit to siraben/nixpkgs that referenced this pull request Oct 5, 2020
The problem was introduced by commit 97a32bc (within PR NixOS#79696).
nixos/tests/common/ec2.nix:6:17 called with unexpected argument 'meta'
dawidsowa added a commit to dawidsowa/nixpkgs that referenced this pull request Oct 11, 2020
The problem was introduced by commit 97a32bc (within PR NixOS#79696).
nixos/tests/common/ec2.nix:6:17 called with unexpected argument 'meta'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.