Proposal/RFC: support runtime extension of format file search path (permit adding directories other than <store-path>/share/nixos-generator)

[tomeon]

nixos-generators-search-path-proposal

Proposal

Support runtime extension of the format file search path via environment variables and CLI options.

Motivation

Provide a mechanism for end users and third parties to host libraries of nixos-generate formats.

Hypothetical usage

$ NIXOS_GENERATORS_FORMAT_SEARCH_PATH=/some/location:/another/location \
    nixos-generate \
        --format-search-path /yet/another/location \
        --format-search-path /yes/there/is/another/location \
        --format my-custom-format \
        --flake '.#my-host'

This would make nixos-generate search for the format file my-custom-format.nix in the following directories, ordered from highest to lowest precedence:

1. /yes/there/is/another/location
2. /yet/another/location
3. /some/location
4. /another/location
5. /nix/store/...-nixos-generators/share/nixos-generator/format

In other words:

1. CLI arguments are highest-precedence, with later arguments (those toward the end of argv) being of higher precedence than earlier arguments,
2. The search path specified in NIXOS_GENERATORS_FORMAT_SEARCH_PATH is next-highest-precedence, and
3. The builtin format directory is lowest-precedence.

Example use cases

• Setting the NIXOS_GENERATORS_FORMAT_SEARCH_PATH environment varible in a devshell configuration so that entering nix develop makes the project's custom nixos-generate formats available via the --format flag rather than only via typing out their paths.
• Using makeWrapper/wrapProgram to ship a custom nixos-generate with additional format definitions, without having to rebuild nixos-generators itself (and modify its builder, etc.).
• Easily incorporating libraries of custom formats supplied by third parties.

WDYT?

I would be happy to submit a PR introducing the described changes if they sound sane and desirable. Thanks in advance for your feedback.

[Lassulus]

well, I guess it wouldn't do any harm to have multiple search paths? I usually just type the full path to the format, so it's not really a usecase for me. however, if the implementation is not to complex I would be happy to merge it

[tomeon]

Cool, thank you -- I'll put something together, minding to KISS.

[Pacman99]

I have another idea that could solve a similar problem. Maybe --format could accept a flake uri, so you could do nixos-generate --flake .#NixOS --format github:divnix/digga#nixosModules.bootstrapIso. And maybe a new flake output could be used, something like generatorModules, since formats aren't exactly nixos modules.

well, I guess it wouldn't do any harm to have multiple search paths? I usually just type the full path to the format, so it's not really a usecase for me. however, if the implementation is not to complex I would be happy to merge it

I think the main issue we're facing is the usage of custom formats that can't be added to nixos generators proper, but we would like users to be able to use them easily. Since the formats are in remote repositories, its hard to get the exact path to them.

[blaggacao]

Smells like a use case for the nix bundler interface...