Per nixosConfiguration pkgs instances?

[lpchaim]

This isn't really an issue per se, feel free to close if it doesn't feel appropriate. I'm basically just wondering if this use case I've come up with is possible and looking for guidance. For all I know I might just be missing something simple!

Anyway, I have the following snippet on my flake:

  outputs = inputs:
    inputs.snowfall-lib.mkFlake {
      inherit inputs;
      src = ./.;

      channels-config = {
        allowUnfree = true;
        cudaSupport = true;
      };
    };

Note I have cudaSupport enabled on my channels-config, since my main rig has a NVIDIA GPU. Say I have another machine with an AMD GPU and would like to also add rocmSupport = true as well.

That does in fact work and builds just fine, but I've stumbled upon some services such as ollama which treat cuda and rocm support as mutually exclusive, so when both are enabled it spits out a warning and defaults to none by default. Not really a big deal since I can just have modules override that manually, but it does result in a somewhat inconsistent configuration without manual intervention and I figure it builds packages with unnecessary support for cuda/rocm the actual hardware may not support.

Can I perhaps wire something like that up on my end? Maybe somehow build a pkgs instance by hand and set specific configurations to use it, or specialized overlays I'd selectively apply? I've read the docs and messed around but haven't been able to come up with much, at least not without bypassing snowfalllib and building said configurations by hand or ugly hacks I'd like to avoid if at all possible.

[jakehamilton]

I don't currently have any plans to support this in Snowfall Lib. I think the pattern that NixPkgs uses for this kind of things is fundamentally wrong and is the reason issues like this arise. One thing you could try is using pkgs.extend (final: prev: {}) to apply a modification to the package set, producing a new one. Not sure if that will allow you to set config.cudaSupport, but if it does then having specific package sets like this could work well if added via overlays.

[lpchaim]

Gotcha, that's perfectly valid. Thank you for elaborating.

Question, after doing some thinking, would you be open to a PR that implements something like

channels-config = { allowUnfree = true; }; # Existing config option
channels.amd.config = { rocmSupport = true; }; # New, per-channel option

which in flake-utils-plus.lib.mkFlake then maps to

channels.amd.config = { rocmSupport = true; };

I'm assuming that doesn't break internal logic too much, of course, I could be completely off-base.

[jakehamilton]

You should already be able to pass these args, non-snowfall lib values are passed to flake-utils-plus iirc. Though you won't have a way to consume these different channels by default. I think you would have to use the library's helpers like create-system directly so you can pass along specific channels.

[lpchaim]

Ah, I see. I assumed that didn't work because I tried and it didn't seem to change anything, I'll give what you said a try.
Thanks for your help, I'll go ahead and close this now.

[anntnzrb]

Fixed

Per-nixosConfiguration pkgs instances are now supported!

Solution

Two changes enable this:

1. Per-channel configuration:
You can now define channel-specific configs:

channels.nixpkgs-cuda.config = {
  allowUnfree = true;
  cudaSupport = true;
};

channels.nixpkgs-rocm.config = {
  allowUnfree = true;
  rocmSupport = true;
};

2. Per-host channel selection:
Assign different channels to different hosts:

systems.hosts.nvidia-desktop.channelName = "nixpkgs-cuda";
systems.hosts.amd-laptop.channelName = "nixpkgs-rocm";

Complete Example

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    nixpkgs-cuda.follows = "nixpkgs";
    nixpkgs-rocm.follows = "nixpkgs";
    snowfall-lib.url = "github:snowfallorg/lib";
  };

  outputs = inputs:
    snowfall-lib.mkFlake {
      inherit inputs;
      src = ./.;

      channels-config.allowUnfree = true;
      
      channels.nixpkgs-cuda.config = {
        allowUnfree = true;
        cudaSupport = true;
      };
      
      channels.nixpkgs-rocm.config = {
        allowUnfree = true;
        rocmSupport = true;
      };

      systems.hosts.nvidia-desktop.channelName = "nixpkgs-cuda";
      systems.hosts.amd-laptop.channelName = "nixpkgs-rocm";
    };
}

Each host gets its own pkgs instance with the appropriate GPU support, eliminating conflicts with services like ollama.

Closes #128
Commit: anntnzrb/snowfall-lib@d1eb5eb