Help with disko

[mariolopjr]

Hi Vic!! Appreciate you providing den, flake-file, and even vix to make such a straightforward configuration for NixOS and all the things!! Been redoing my configuration to fully embrace dendritic and den but getting hung up on how exactly to rewrite/fit in my disko configuration.

Normally, in a flake modules kind of way, I would normally have this as a flake module

{inputs, ...}: {
  flake.modules.nixos."hosts/winterfell" = {
    imports = [
      inputs.disko.nixosModules.disko
    ];

    disko.devices = {
      disk.main = {
        type = "disk";
        device = "/dev/nvme0n1";
        content = {
          type = "gpt";
          partitions = {
            boot = {
              name = "boot";
              size = "1M";
              type = "EF02";
            };
            esp = {
              size = "512M";
              type = "EF00";
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/boot";
                mountOptions = ["defaults"];
              };
            };
            swap = {
              size = "16G";
              content = {
                type = "swap";
                randomEncryption = true;
              };
            };
            luks = {
              size = "100%";
              content = {
                type = "luks";
                name = "cryptroot";
                settings = {
                  allowDiscards = true;
                };
                content = let
                  mountOptions = [
                    "compress=zstd"
                    "ssd"
                    "noatime"
                    "discard=async"
                  ];
                in {
                  type = "btrfs";
                  extraArgs = ["-f"];
                  postCreateHook = ''
                    MNTPOINT=$(mktemp -d)
                    mount "/dev/mapper/cryptroot" "$MNTPOINT" -o subvol=/
                    trap 'umount $MNTPOINT; rm -rf $MNTPOINT' EXIT
                    btrfs subvolume snapshot -r $MNTPOINT/root $MNTPOINT/root-blank
                  '';
                  subvolumes = {
                    "/root" = {
                      mountpoint = "/";
                      inherit mountOptions;
                    };
                    "/persist" = {
                      mountpoint = "/persist";
                      inherit mountOptions;
                    };
                    "/nix" = {
                      mountpoint = "/nix";
                      inherit mountOptions;
                    };
                  };
                };
              };
            };
          };
        };
      };
    };
  };
}

I tried converting it in various ways, first as a non-dedritic module with the exact same file contents, but couldn't figure out how to have disko automatically set the filesystems

winterfell.nix

disko.nix

It doesn't help that my understanding of the Nix language is at a beginner level. Definitely running before I can walk but I just really like the simplicity that den offers, just unable to find examples of utilizing flakes like disko or nvf and how to appropriately convert them to fit this paradigm.

If this is not the proper forum, or I did not provide enough information, let me know! I'll continue experimenting as well :)

As an aside -- am I missing examples in the repo somewhere on converting external flakes like nvf, disko, etc. into the dendritic pattern? I looked at https://github.com/vic/den/tree/main/templates/default/modules/_example but couldn't find a good example of converting some of these flakes.

Appreciate it!! :)

[vic]

Hey, @mariolopjr

I know nothing about disko, but I will try to be of help anyways.

First of all, lets assume that disko.devices = ... is the whole configuration tree you shared, no need to repeat your particular disko configuration.

Now, I might be wrong, so please don't trust me, since I have zero experience with disko. But this is what I imagine from reading the documentation (at flake-parts, I'm following this since I see you are importing their disko flakeModule).

# modules/disko.nix -- this is a flake-parts module, like those used in `den` and any dendritic.
{
  imports = [ inputs.disko.flakeModules.default ];
  flake.diskoConfigurations.my-pc = {
    disko.devices = ...; # all your large disko config.
  };
}

according to the documentation, this is all you need for using with their CLI tools like disko and disko-install, there's nothing den specific here, since den is about configuring nixos hosts. However ....

Digging a bit more in the disko repo I see there's also the option of configuring disko inside a nixos configuration, and that might be an actual usable example in den. I don't know if that is what you need, since your code looks like solved by my previous snippet.

The following is den configuration, assuming you already defined den.hosts.<system>.my-pc somewhere.

# modules/my-pc/partitioning.nix -- lets configure disk inside my-pc nixos.
{ inputs, ... }:
{
  den.aspects.my-pc.nixos = {
    # anything inside here is a normal NixOS configuration module.
    # so you can follow any example that mentions NixOS configurations.
    
    # NOTE: we are importing a `nixos` module not the flake one like on our previous example
    # since this block is a nixos configuration.
    # This is the module being loaded: https://github.com/nix-community/disko/blob/master/module.nix
    imports = [ inputs.disko.nixosModules.default ];
   
    # the module defines options.disko, so I assume after importing the nixos module you can do:
    disko.devices = {
      # everything you already had here.
    };
  };
}

I dont know which of the two examples is more useful, it depends if you want to use the disko CLI tools use the first option that expose diskoConfigurations as output of your flake. If not, the second example setups disko inside your my-host nixosConfiguration, and I have no clue what that does. Hahaa.

Anyways, hope this can be of any help. Feel free to ask if you have more questions. Also if you are willing to send a PR that adds a very minimal disko setup to one of your example hosts, fell free to sen the PR it will be more than welcome!

[vic]

Shorter answer: just rename flake.modules.nixos."hosts/winterfell" to den.aspects.winterfell.nixos :)

[mariolopjr]

Thank you so much!! That worked and the CLI still works since the disks can be exposed via flake.diskoConfigurations or via flake.nixosConfigurations.

And what this module does is it allows to declare disks in a manner that a blank, unformatted system can have its disk formatted with the same disk configuration it will use to boot with. No more manual partitioning!

Also if you are willing to send a PR that adds a very minimal disko setup to one of your example hosts, fell free to sen the PR it will be more than welcome!

This is a great idea! I will refine (and simplify) the disko configuration to include an example den aspect in the repo! Thank you again, the walk thru steps really helped me understand the process of den-ifying other modules lol

[mariolopjr]

Follow-up question @vic (sorry!).

I'd like to make the configuration more modular and less brittle (esp once I open a pull request to add the example here). Right now, I have to duplicate the exact same configuration with minor changes between two systems (the VM and non-VM, but in the future, most systems will use a similar configuration but with some options tweaked if needed).

So now I have a disko.nix configuration for the hardware system and a disko-vm.nix for the VM. I linked and included the relevant code sections below where you can see the only values that change.

In this case, I need to change the device to /dev/vda for the VM, reduce the swap size to 4G, and add the option imageSize = "40G"; to disk.main.

I tried all sorts of ways to de-duplicate this effort with mkOption, passing in arguments, etc., but it didn't seem to work and a lot of it didn't feel dendritic. How do you approach reusable modules without importing and passing specialArgs? I've tried creating infra.disko but that seemed incompatible with the values under .nixos.

The other complication is this type of filesystem management should be opt-in for the systems that provide it in the aspect. Really appreciate any advice you can give, thank you!! :)

disko.nix

{
  inputs,
  ...
}:
{
  den.aspects.winterfell.nixos = {
    imports = [
      inputs.disko.nixosModules.default
    ];

    disko.devices = {
      disk.main = {
        type = "disk";
        device = "/dev/nvme0n1";
        content = {
          type = "gpt";
          partitions = {
            boot = {
              name = "boot";
...SNIP
            swap = {
              size = "16G";
              content = {
                type = "swap";
                randomEncryption = true;
              };
            };
...SNIP

disko-vm.nix

{
  inputs,
  ...
}:
{
  den.aspects.winterfell-vm.nixos = {
    imports = [
      inputs.disko.nixosModules.default
    ];

    disko.devices = {
      disk.main = {
        type = "disk";
        device = "/dev/vda"; # changed for VM
        imageSize = "40G"; # added for VM
        content = {
          type = "gpt";
          partitions = {
            boot = {
              name = "boot";
...SNIP
            swap = {
              size = "4G"; # changed for VM
              content = {
                type = "swap";
                randomEncryption = true;
              };
            };
...SNIP

[vic]

The module system will merge whatever config tree you need:

{ den, ...}:
 
  den.aspects.winterfell.includes = [
     den.aspects.disko._.common
     den.aspects.disko._.machine
  ];

  den.aspects.winterfell.includes = [
     den.aspects.disko._.common
     den.aspects.disko._.vm
  ];

  den.aspects.disko._.common = {
     nixos.disko.devices = { ... } # everything except parts that change
  };

  den.aspects.disko._.vm = {
     nixos.disko.devices = {
       disk.main.content.partitons.swap = { ... } # specific to VM
     };
  };

  # same for hardware machine.
}

[mariolopjr]

Oh oof that does make sense. Still understanding the smilie ._. faces haha! Thank you again, will go back and modularize the config with this!

[vic]

Be sure to checkout the latest release notes (see migration notes) and the README, it has more documentation about those faces. :)

[mariolopjr]

Super huge release, thank you! Will def dive in and migrate my config. Thanks a lot, it looks like it simplifies a lot!!

[vic]

Still understanding the smilie ._. faces haha!

@mariolopjr take a look at this: diff

Note

Experimental angle-brackets FEATURE #38
Allows writing <foo/bar/baz> instead of den.aspects.foo.provides.bar.provides.baz or even den.aspects.foo._.bar._.baz, at least for read access of very nested trees.

[mariolopjr]

Excited for the angle brackets, I feel it will simplify and make things look more straightforward! Appreciate all the hard work you've into den, it's really coming together and my favorite organization framework I've seen for Nix :)

[mariolopjr]

Hi @vic I'm back! (hope you're not sick of my questions!!)

Been re-working my small repo to den 0.3 and it looks like I got it mostly sorted, however, disko rears it's hard-to-configure head. In this case, I think it's directly related to the parameterized aspects feature den 0.3 has for the includes.

Here's the actual error (and the PR I gave in progress: https://github.com/mariolopjr/infra/pull/1). This is the new disko file that's been parameterized, and using it in the host includes: https://github.com/mariolopjr/infra/blob/d71305478c9d455e7e84af042c5039435e32f58c/modules/hosts/winterfell/winterfell.nix#L20

error: The option `disko.devices.disk.main.content.partitions.ESP.content.mountpoint' is defined multiple times while it's expected to be unique.

       Definition values:
       - In `/nix/store/xazxk1swrrkc974lhcrmdcprz914sqk1-source/modules/namespace.nix, via option den.aspects.profile.provides.disko.<function body>.includes."[definition 1-entry 4]".nixos': "/boot"
       - In `/nix/store/xazxk1swrrkc974lhcrmdcprz914sqk1-source/modules/namespace.nix, via option den.aspects.profile.provides.disko.<function body>.includes."[definition 1-entry 4]".nixos': "/boot"
       Use `lib.mkForce value` or `lib.mkDefault value` to change the priority on any of these definitions.

I've tried digging in with nix repl, and it looks like it's trying to set the whole disko.devices (maybe once for { host } and once for { host, user })? I tried specify host within the aspect, and within the winterfell aspect via inherit, and all the things but cannot seem to not have this configuration duplicated.

Sorry to constantly bug you, was just curious if you have any thoughts knowing the inner workings of the new includes system.

Previously, I set the disko configuration under den.aspects.winterfell.nixos, but doing that in den 0.3 also gave me the is defined multiple times error.

Appreciate any help in advance!

[vic]

Ok I think I have an idea of how to fix it (using different contexts) so default is not included in the user the two additional times. The test must verify that only one hello package is configured in the host.

[vic]

Update for context in this thread: It works now, the PR is green on integration tests (nixos/darwin) but we still have some unit-test to fix.

[mariolopjr]

Wooo thank you so much!! Looking forward to it! Also, #45 is looking really good too. I've been evaluating my configuration and trying to make it easier for folks to just copy/paste and have it work in their setup (flake-file makes that so easy too)

[vic]

Wooo thank you so much!! Looking forward to it! Also, #45 is looking really good too. I've been evaluating my configuration and trying to make it easier for folks to just copy/paste and have it work in their setup (flake-file makes that so easy too)

Yep, I have some plans regarding collaboration, and NOT copy-pasting. But actually using modules directly from others repositories (that was why I created dendrix, as an AUR-like list of dendritic repositories) However, for that to work I had to created flake-file, so each module can specify its requirements. So people dont need to depend on my whole flake (nor in all my inputs) if they just want to use, say my doom-emacs configuration, that sub-directory of modules define its own inputs. flake-aspects and den allow us to create functions that people can include in their hosts aspects, to configure them. stay tuned for more on that ! After den is stable enough, I'll be rewriting my whole infra so I can extract bits into denful, and of course you are invited to submit generic aspects in denful, that's the idea, it will be like a vim-distribution. But I still have to design how "facet" modules in denful will look. For example, in denful you will be able to have "tiling-wm" that provides "hyprland" or "Niri" or also that contributes to another anarchy aspect (a pun in omarchy). So features like tiling-wm can be used independently and they can also register themselves, for example in anarchy.provides.desktop, so if you enable tiling-wm it contributes to anarchy desktop. Anyways, still have to think about how that will look. One step at a time.

[mariolopjr]

That's definitely cool, will stay tuned! :D

[xdbr]

The last couple of days I have closely followed the rapid development on your amazing set of libraries, @vic and I am utterly amazed! However, I still struggle to include disko and individual disk partitions as aspects... Particularly with the latest developments (which I absolutely think are the way to go!) I'm still a little confused and just can't get it to work (yet)... The documentation is very dense for me and the concepts behind it very high-level...

Did someone already use the latest developments and can provide a minimal example of how to include disko, sops-nix or other flake modules? Or maybe even has an infrastructure config to share? Any help or examples are much appreciated!

[vic]

Hey, thanks :) it's been a long road actually, I've been working on these libraries like almost for a year (I believe vic/checkmate and vic/import-tree are almost a year old now), and others have been born as I had time. I'm quite happy that people have something they can use now, (den -- other libs being supporting den). So, thanks to you for trying out den. I pretty much appreciate people using it.

Looks like disko is very popular along with facter, I did not know of it until recently, haha, I do use sops-nix, but maybe these two could be good examples, and more than examples I'd like to provide some modules for easily integrating them, but that would be in vic/denful. denful will be the place for more advanced batteries than the basic ones found in den itself. I'm still finishing other things tho, and pretty much time limited.

Regarding documentation, yes, my intention is that the main readme section should be sufficiently clear for creating most-common nixos/darwin/home configurations. I wanted it to be just one-page of declaring a host and adding nixos configs to its aspect, and that's it. Any help making the docs more minimalistic yet sufficiently clear is more than welcome. I'm not an english native speaker, and sometimes I just write too fast and forget about typos and such, so help is very much appreciated.

And then other sub-sections expand on details, I'm currently working on WIP: documentation about functor-aspects that are the functional core of den. People do not need to understand them for using den, but if they do, they can unlock the same power as den itself uses.

So, given that I'm not a disko user (and I don't have the time to come up with full repo example) maybe the best option is to follow this thread and maybe other people can help.

Just remember that the basics of aspects are these:

{
  den.aspects.my-laptop.nixos = {
    # whatever NixOS options here.
    # large nixos modules can be imported
    # (underscore for import-tree to ignore from top-level flake-modules auto-import)
    imports = [ ./_disko.nix ];
  };
}

If people are willing to contribute another template, that focuses on having a single nixos configuration with sops/disko/facter it would be gladly accepted (we just need the CI to check that template as it does others that we have now)

[vic]

Maybe I can help if you expand on what makes you feel confused, it's not my intention that the documentation confuses people, on the contrary, otherwise, the docs are not doing their job. So help me identifying what is confusing to you so we can improve docs.

[xdbr]

Hey @vic thanks for your super responsive and quick reply – I see you are wholeheartedly invested and that's a very valueable thing :)

I have read by and far through all the other issues, discussions, repos, and I'll give it another shot tomorrow and see what I can come up with. Maybe I can prepare a little repo and we can go from there. I would just like to say I really appreciate your work as it is exactly how I imagine this ecosystem to be and where I would like to see it go – if I can say this, since I am still very early in my nix expertise. Keep up the good work!

[vic]

if that is possible and what namespace are actually for.

exactly, namespaces give you a new name to refer to an aspect tree. cool thing about them is that it abstracts where specifically are these aspects from.

# in your local flake
{ fleet, ... }:
{
  fleet.stuff.nixos = { ... };
}

If I was to re-use some of your modules, I could set:

# my flake that consumes your aspects
{ inputs, ... }:
{ 
  _module.args.fleet = inputs.xdbr.denful.fleet; # or from somewhere else
}

and your modules would "work" since they have no den.aspects.<your-machine> specifics.

[xdbr]

Hey @vic, good catch about ssh.nix!
I was already wondering why the duplicated definitions of openssh.enable between https://github.com/xdbr/fleet/blob/dev/modules/hosts/defaults/ssh.nix and
https://github.com/xdbr/fleet/blob/dev/modules/hosts/defaults/nixos/services/ssh.nix do not clash but now that you point it out, it makes perfect sense – thank you!

My original idea with defining ssh below den.default (at https://github.com/xdbr/fleet/blob/dev/modules/hosts/defaults/ssh.nix#L3) was that it would apply to all hosts, no matter if it is a nixos or darwin system. But i guess this might be a place where a parametric aspect would come in handy? Not exactly sure yet, but I will investigate!

[vic]

Nice, just be careful when adding parametric aspects in den.default (not everything needs to be in default, it is easy to get duplicates if you don't know what you are doing), having said that, den.default serves as a aspect-configuration router, it was made for that, but be sure to read and understand about how contexts and dependencies work in Den, be sure to read the documentation.

[xdbr]

OK cool – you see, I'm still learning! I have checked out the new documentation from front to back and must say: congratulations! It surely will make the adoption a lot lot easier for new people coming to this project!

Regarding the application of the fleet namespace: it generally works, but for some reason I'm not able to apply it in https://github.com/xdbr/fleet/blob/dev/modules/hosts/barbara.bar/configuration.nix ...

It works aboslutely fine and as expected in https://github.com/xdbr/fleet/blob/dev/modules/hosts/barbara.bar/networking.nix, https://github.com/xdbr/fleet/blob/dev/modules/hosts/barbara.bar/hardware-configuration.nix, as well as https://github.com/xdbr/fleet/blob/dev/modules/hosts/barbara.bar/nginx.nix but not in https://github.com/xdbr/fleet/blob/dev/modules/hosts/barbara.bar/configuration.nix.

Might it have to do with the usage of __findFile and the angle-bracket syntax? Or is it something entirely different? Any ideas?

[vic]

Can you open another discussion for the namespace thing ? let's explore it there.

EDIT: #79 (comment) you uncovered a bug :) Fix is on the way.