passthru.tests but for modules

[roberth]

Describe the problem

• Connection between module and test is not always clear.
• It's unclear how to test without VM tests. Some modules can and should be unit tested.
• No way of figuring out which tests are relevant for a given configuration

Proposal

Use the module meta attribute to determine which tests exists and are applicable.

A new option in meta could work similar to the meta.maintainers module option in that it preserves the path of the defining module(s), which could be helpful for assigning unique names. Alternatively, we could just require module authors to enter a unique name for each test.

{ config, pkgs, ... }:
{
  imports = [ ];
  options = { };
  config = { };
  meta.maintainers = [ ]; # already exists
  meta.tests.foo = pkgs.callPackage ./test.nix { };
}

The last item, "No way of figuring out which tests are relevant for a given configuration" also needs a minimal module list in order to make this solution work. Otherwise, you'll always get all the tests.

Notify maintainers

Metadata

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module: