WIP Add nixos-tests module

[esselius]

Convenience module to make the nixos tests learning curve less steep :)

[roberth]

Hi @esselius,
I'm a little hesitant to merge this as an official-looking "nixos-tests" module, but maybe that's best resolved by prefixing all names with their repo owner anyway. Wdyt?

Also I think it'd be good use module arguments instead of putting functions to modules in files. You could also factor out some of the let bindings so that users aren't stuck with unnecessary limitations while their use case becomes more advanced.
Maybe also rename path to directory so that it's a bit more explicit and the name hints how it's used?

You'd then have options like

• nixosTests.directory: sufficient for basic use, and sets nixosTests.tests.<name>.module
• nixosTests.defaults.args: sets arguments for all tests, propagates to nixosTests.tests.<name>.args
• nixosTests.defaults.module: module that's always "imported"; propagates to nixosTests.tests.<name>.module
• nixosTests.<name>: as already hinted

I don't think runNixOSTest lets you set specialArgs immediately, so that could be a problem. You could approximate it by setting the _module.args option in *.module, but that's not good enough if users want to use their custom arguments in imports. This should be fixed in nixpkgs, e.g. by adding a second function runNixOSTest', and I'd be happy to review it.
Alternatively, it could be bypassed by importing nixos/lib, but that's kind of tricky because we don't really have a nixpkgs flake; only a pkgs, and pkgs.path might make a copy of Nixpkgs in the store (I'd have to check the state of things, and NixOS/nix#6530, which may either fix it, or still break making this more relevant).

Another addition I would recommend is nixosModules.readOnlyPkgs. pkgs.runNixOSTest usually reuses the pkgs, but it won't catch "abuse" of nixpkgs.config for example, as those definitions would just be ignored silently.
Perhaps this could be a flag like nixosTests.tests.<name>.readOnlyPkgs, default = true;?

[esselius]

@roberth

I'm reconsidering my nixos test approach, having a more python-native style with pytest as a possible entrypoint would be nice and possibly simplify test maintenance, but I still haven't reached a point where that's true and work on both darwin/linux yet :)

Thanks for all the great feedback, will surely incorporate this into my next attempt! :)