Policy based custom classes for Impermanence support in aspects

[tschan]

Hi there,

I'm currently trying to wrap my head around the architecture patterns in den to see if they might be helpful for reorganizing my NixOS config.

One aspect (see what I did there 😀) that I find most interesting would be the ability for individual aspects to provide the file paths for Impermanence that belong to the given program the aspect enables. I imagine something like this:

{
  den.aspects.my-tool = {
    homeManager ={
        programs.my-tool.enable = true;
    };
   persistUserFiles = [
     ".config/tool.rc"
   ];
   persistUserFolders = [
     ".config/tool"
   ];
   persistFiles = [
     "/etc/tool.rc"
   ];
   persistFolders = [
     "/etc/tool"
   ];
  };
}

Ideally there is only one additional aspect that then enables Impermanence for a given host and takes these values into account to configure

{
  environment.persistence."/persistence" = {
    # for persistFolders
    directories = [
      ...
    ];
    # for persistFiles
    files = [
      ...
    ];
    users.$USER = {
      # for persistUserFolders with the correct username of the entity the aspect applied to
      directories = [
        ...
      ];
      # for persistUserFiles
      files = [
        ...
      ];
    };
  };
}

I've seen different concepts in the den documentation that would seem a possible avenue for realizing this functionality. Quirks, policies and batteries.forward. I've read in other discussions that policies seem the future for enabling this kind of thing.

So what would be the most idiomatic, concise and modern way to implement this requirement?

[amanako]

Thought I'd share my findings. I suppose you are somewhat familiar with impermanence and should understand den overall. I followed README and examples and that was generally enough to create the class. I currently use den.batteries.forward in this manner (currently all in one file though could be broken up, sry if it's a bit messy).

{
  inputs,
  den,
  lib,
  ...
}: let
  persistClass = {
    host,
    user,
    ...
  }:
    den.batteries.forward {
      each = [
        "host"
        "user"
      ];
      fromClass = target: "persys${lib.optionalString (target == "user") "User"}";
      intoClass = _item: "nixos";
      intoPath = target:
        [
          "environment"
          "persistence"
          host.impermanence.persistenceDir
        ]
        ++ lib.optionals (target == "user") [
          "users"
          user.userName
        ];
      adaptArgs = {config, ...}: {
        osConfig = config;
      };
      guard = {options, ...}: options ? environment.persistence;
    };
in {
  flake-file.inputs.impermanence.url = "github:nix-community/impermanence";

  # Workaround for config.includes not working.
  # From my experience it should be avoided since it fails.
  # Using it causes error for this aspect:
  # error: attribute 'persistence' missing
  den = {config, ...}: {
    schema.host.options.impermanence = let
      defaultPersysDir = "/nix/persist/system";
    in
      lib.mkOption {
        type = lib.types.submodule {
          options.enable = lib.mkOption {
            type = lib.types.bool;
            # If default persistenceDir is changed suppose host wants the module
            default = config.impermanence.persistenceDir != defaultPersysDir;
            description = "Whether to enable impermanence module";
          };

          options.persistenceDir = lib.mkOption {
            type = lib.types.str;
            default = defaultPersysDir;
            description = "Directory for impermanence persistent storage";
          };
        };
      };

    schema.host.includes = [den.aspects.impermanence];

    aspects.impermanence = {host, ...}: {
      includes = [persistClass];
      nixos = {
        imports = [inputs.impermanence.nixosModules.impermanence];
        # This should be minimal for a truly pure setup
        fileSystems."${host.impermanence.persistenceDir}".neededForBoot = true;
      };
    };
  };
}

I use similar pattern for disko. Supposing you use flakes. I use flake-file and highly recommend using it if you aren't already.
To summarize:

• add a flake input (generally best to be as close to place where input is actually referenced)
• add a class in let expression and include it in den.aspect.impermanence to achieve the job
• expose options for host to opt-in
• include the aspect in host schema and use it in other aspects under persys and persysUser classes

In case you are confused, take a look at examples below.

Class does the following:

Creates two custom classes via each - one named persys and other persysUser since they forward to 2 different paths within nixos class: persys to environment.persistence.${host.impermanence.persistenceDir} and persysUser just appends users.${user.userName} to capture attrset for users.

I also used guard to simply verify environment.persistence is present since importing the module should add that option and I want to know it is actually imported. adaptArgs i just copied from some of examples on https://den.denful.dev/guides/custom-classes/ but shouldn't be necessary I think.

Aspect pulls in impermanence nixosModule and sets up some basic configuration.

Regarding "workaround" comment, splitting options and includes should solve config.includes error but I
wanted to use them together since I didn't want file to be too small.
From there just:

1.Host enables with:

  den.hosts.x86_64-linux.nebula = {
    # Could be explicit: impermanence.enable = true;
    # Changing default directory implicitly enables module
    impermanence.persistenceDir = "/persist";
  };

2. Use it in aspects like:

  den.default = {
    persys = {
      hideMounts = true;
      directories = [
        # Without this dir all users/groups without specified
        # uids/gids will have them reassigned on reboot.
        "/var/lib/nixos"

        # Popup lecturing on sudo usage
        "/var/db/sudo/lectured"

        # Time stamps for systemd tasks which should help with remembering timers countdown
        "/var/lib/systemd/timers"
      ];

      files = [
        # Fix wpa/network errors
        "/etc/machine-id"
      ];
    };
...
};

and examples for some user stuff:

{
  den.aspects.shell.zoxide = {user, ...}: {
    persysUser.directories = [
      ".local/share/zoxide" # database for previous entries
    ];

    homeManager.programs.zoxide = {
      enable = true;
      enableFishIntegration = true;
    };
  };
}

or for the user itself:

  # My current users name
  den.aspects.lunar-scar = {
    persysUser.directories = [
      "Dev"
      "Documents"
      "Downloads"
      "nix-config" # Main config
      "Pictures"
      {
        directory = "keys/ssh";
        mode = "0700";
      }
      {
        directory = "keys/gnupg";
        mode = "0700";
      }

      # For direnv to remember allowed .envrc files
      ".local/share/youtube-tui"
      ".local/share/systemd/timers"
    ];
...
}

This way you don't need to pass user.userName each time around as it's infered from context(aspects could be static - that is not take {user, ...} unlike zoxide example above but I thought being explicit is better.
Overall, kinda messy at the moment but should help you get overall point.

To answer your question from my understanding, using quirks here is possible but I think they should be used for smaller simple stuff like firewall ports in the examples.

I suppose policies are prefered (forward battery is implemented via them) and I will likely switch over to them but this config is made from before policies were introduced. One possible downside is that policies require being enabled explicitly while batteries just get enabled, but on the other hand that difference could be used. I currently more like this opt-in-via-options approach.

Also I see you mentioned using 4 classes for each files and directories but that seems overkill and accessing them as members of attribute set with .directories or .files should suffice.

Hopefully you got the hang of it. If I was unclear on some point feel free to point it out. And if anyone else more experienced reads this feel free to point out if I am mistaken somewhere.

[tschan]

Thank you very much for your input. I tried to adapt your code a bit to my needs and I came up with this:

{
  inputs,
  den,
  lib,
  ...
}:
let
  persistClass =
    { host, user, ... }:
    den.batteries.forward {
      each = [
        "host"
        "user"
      ];
      fromClass = target: "persist${lib.optionalString (target == "user") "User"}";
      intoClass = _item: "nixos";
      intoPath =
        target:
        [
          "environment"
          "persistence"
          host.persistenceDir
        ]
        ++ lib.optionals (target == "user") [
          "users"
          user.userName
        ];
      adaptArgs =
        { config, ... }:
        {
          osConfig = config;
        };
      guard = { options, ... }: host.persistenceDir != "";
    };
in
{
  den =
    { config, ... }:
    {
      schema.host.options.persistenceDir = lib.mkOption {
        type = lib.types.str;
        default = "";
        description = "Directory for impermanence persistent storage (enables Impermanence for host if set)";
      };

      aspects.impermanence =
        { host, user, ... }:
        {
          includes = [ persistClass ];

          nixos = lib.optionalAttrs (host.persistenceDir != "") {
            imports = [ inputs.impermanence.nixosModules.impermanence ];

            fileSystems."${host.persistenceDir}".neededForBoot = true;

            boot.initrd.systemd.services.recreate-root = {
              description = "Rolling over and creating new filesystem root";

              wantedBy = [ "initrd.target" ];
              requires = [ "initrd-root-device.target" ];
              after = [
                "initrd-root-device.target"
                "local-fs-pre.target"
              ];
              before = [
                "sysroot.mount"
                "create-needed-for-boot-dirs.service"
              ];

              unitConfig.DefaultDependencies = "no";
              serviceConfig.Type = "oneshot";

              script = ''
                mkdir /btrfs_tmp
                mount /dev/mapper/cryptroot /btrfs_tmp
                if [[ -e /btrfs_tmp/root ]]; then
                  mkdir -p /btrfs_tmp/old_roots
                  timestamp=$(date --date="@$(stat -c %Y /btrfs_tmp/root)" "+%Y-%m-%d_%H:%M:%S")
                  mv /btrfs_tmp/root "/btrfs_tmp/old_roots/$timestamp"
                fi

                delete_subvolume_recursively() {
                  IFS=$'\n'
                  for i in $(btrfs subvolume list -o "$1" | cut -f 9- -d ' '); do
                    delete_subvolume_recursively "/btrfs_tmp/$i"
                  done
                  btrfs subvolume delete "$1"
                }

                for i in $(find /btrfs_tmp/old_roots/ -maxdepth 1 -mtime +30); do
                  delete_subvolume_recursively "$i"
                done

                btrfs subvolume create /btrfs_tmp/root
                umount /btrfs_tmp
              '';
            };

            users.mutableUsers = false;
            users.users = {
              root.hashedPasswordFile = "${host.persistenceDir}/passwords/root";
              "${user.userName}".hashedPasswordFile = "${host.persistenceDir}/passwords/${user.userName}";
            };
          };
        };
    };
}

I import the impermanence aspect into my base system aspect and it works almost perfectly.

A host like this works as expected:

{
  den.hosts.x86_64-linux.host = {
    persistenceDir = "/persistence";

    users.user1 = { };
  };
}

nix-repl> nixosConfigurations.host.config.environment.persistence."/persistence".users.user1.directories
[
  { ... }
  { ... }
...

On a host with more than one user it breaks though:

{
  den.hosts.x86_64-linux.host = {
    persistenceDir = "/persistence";

    users = {
      user1 = { };
      user2 = { };
    };
  };
}

nix-repl> nixosConfigurations.host.config.environment.persistence."/persistence".users.user1.directories
error:
       … while evaluating a branch condition
         at /nix/store/3a2vdn5i7vd2wl654xs8nb52jf1v6cbh-source/lib/attrsets.nix:1188:11:
         1187|           name: value:
         1188|           if isAttrs value && cond value then recurse (path ++ [ name ]) value else f (path ++ [ name ]) value
             |           ^
         1189|         );

       … in the left operand of the AND (&&) operator
         at /nix/store/3a2vdn5i7vd2wl654xs8nb52jf1v6cbh-source/lib/attrsets.nix:1188:28:
         1187|           name: value:
         1188|           if isAttrs value && cond value then recurse (path ++ [ name ]) value else f (path ++ [ name ]) value
             |                            ^
         1189|         );

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: The option `environment.persistence' in `nixos@impermanence/<anon>:0' is already declared in `nixos@impermanence/<anon>:0'.

Any ideas?

[amanako]

The issue stems from duplicated paths being forwared to environment.persistence when each is used like this. What really needs to be done is creating separate classes where one forwards on host presence, and second one on user, then no collision occurs. Sorry, I don't currently have multiple users defined so I haven't tested it for that case and thanks for pointing it out. Hopefully didn't cause you any trouble.

  persistHostClass = {host}:
    den.batteries.forward {
      each = lib.singleton true;
      fromClass = _: "persys";
      intoClass = _: "nixos";
      intoPath = _: ["environment" "persistence" host.impermanence.persistenceDir];
      adaptArgs = {config, ...}: {osConfig = config;};
      guard = {options, ...}: options ? environment.persistence;
    };

  persistUserClass = {
    host,
    user,
  }:
    den.batteries.forward {
      each = lib.singleton true;
      fromClass = _: "persysUser";
      intoClass = _: "nixos";
      intoPath = _: ["environment" "persistence" host.impermanence.persistenceDir "users" user.userName];
      adaptArgs = {config, ...}: {osConfig = config;};
      guard = {options, ...}: options ? environment.persistence;
    };

Then just include both in impermanence aspect.
I tried making a new user and it seems to work. Tell me if anything else pops up.

[tschan]

No worries, thank you for trying to help. :) Unfortunately changing it to this does not make any difference:

{
  inputs,
  den,
  lib,
  ...
}:
let
  persistClassHost =
    { host }:
    den.batteries.forward {
      each = lib.singleton true;
      fromClass = _: "persist";
      intoClass = _: "nixos";
      intoPath = _: [
        "environment"
        "persistence"
        host.persistenceDir
      ];
      adaptArgs =
        { config, ... }:
        {
          osConfig = config;
        };
      guard = { options, ... }: host.persistenceDir != "";
    };
  persistClassUser =
    { host, user }:
    den.batteries.forward {
      each = lib.singleton true;
      fromClass = _: "persistUser";
      intoClass = _: "nixos";
      intoPath = _: [
        "environment"
        "persistence"
        host.persistenceDir
        "users"
        user.userName
      ];
      adaptArgs =
        { config, ... }:
        {
          osConfig = config;
        };
      guard = { options, ... }: host.persistenceDir != "";
    };
in
{
  den =
    { config, ... }:
    {
      schema.host.options.persistenceDir = lib.mkOption {
        type = lib.types.str;
        default = "";
        description = "Directory for impermanence persistent storage (enables Impermanence for host if set)";
      };

      aspects.impermanence =
        { host, user, ... }:
        {
          includes = [
            persistClassHost
            persistClassUser
          ];

          nixos = lib.optionalAttrs (host.persistenceDir != "") {
            imports = [ inputs.impermanence.nixosModules.impermanence ];

            fileSystems."${host.persistenceDir}".neededForBoot = true;

            boot.initrd.systemd.services.recreate-root = {
              description = "Rolling over and creating new filesystem root";

              wantedBy = [ "initrd.target" ];
              requires = [ "initrd-root-device.target" ];
              after = [
                "initrd-root-device.target"
                "local-fs-pre.target"
              ];
              before = [
                "sysroot.mount"
                "create-needed-for-boot-dirs.service"
              ];

              unitConfig.DefaultDependencies = "no";
              serviceConfig.Type = "oneshot";

              script = ''
                mkdir /btrfs_tmp
                mount /dev/mapper/cryptroot /btrfs_tmp
                if [[ -e /btrfs_tmp/root ]]; then
                  mkdir -p /btrfs_tmp/old_roots
                  timestamp=$(date --date="@$(stat -c %Y /btrfs_tmp/root)" "+%Y-%m-%d_%H:%M:%S")
                  mv /btrfs_tmp/root "/btrfs_tmp/old_roots/$timestamp"
                fi

                delete_subvolume_recursively() {
                  IFS=$'\n'
                  for i in $(btrfs subvolume list -o "$1" | cut -f 9- -d ' '); do
                    delete_subvolume_recursively "/btrfs_tmp/$i"
                  done
                  btrfs subvolume delete "$1"
                }

                for i in $(find /btrfs_tmp/old_roots/ -maxdepth 1 -mtime +30); do
                  delete_subvolume_recursively "$i"
                done

                btrfs subvolume create /btrfs_tmp/root
                umount /btrfs_tmp
              '';
            };

            # make firejail block access to the impermanence persistence store
            programs.firejail.blockPersistence = true;

            users.mutableUsers = false;
            users.users = {
              root.hashedPasswordFile = "${host.persistenceDir}/passwords/root";
              "${user.userName}".hashedPasswordFile = "${host.persistenceDir}/passwords/${user.userName}";
            };
          };
        };
    };
}

Exact same error as before. I'm a bit surprised that it works for you and not for be. Maybe it makes a difference which aspects define some persistence then? I have definitions all over the place.
In my base system aspect:

{
  den.aspects.sys-preset-basic = {
    persist = {
      directories = [
        ...
      ];
      files = [ ... ];
    };

    persistUser = {
      directories = [ ... ];
      files = [ ... ];
    };
  };
}

In a specific user aspect

  den.aspects.user1 = {
    persistUser = {
      directories = [
       ...
      ];

      files = [
        ...
      ];
    };
  };

And also in the host aspect:

{
  den.aspects.host = {
    provides.to-users =
      { user, ... }:
      {
        persistUser = lib.optionalAttrs (user.userName == "user1") { directories = [ ... ]; };
      };
    persist = {
      directories = [
        ...
      ];
    };
  };
}

[tschan]

Adding some debug tracing to the evaluation like this:

{
  persistClassHost =
    { host }: builtins.trace "persistClassHost: ${host.hostName}"
    den.batteries.forward {};

  persistClassUser =
    { host, user }: builtins.trace "persistClassUser: ${host.hostName}: ${user.userName}"
    den.batteries.forward {};

aspects.impermanence =
        ({ host, user }: builtins.trace "impermance: ${host.hostName}: ${user.userName}" {};
}

I can at least see that the aspect and includes get evaluated twice twice as much if there are two users:

nix-repl> nixosConfigurations.oneuser.config.environment.persistence."/persistence".users.user1.files           
trace: impermance: oneuser: user1
trace: persistClassHost: oneuser
trace: persistClassUser: oneuser: user1
trace: impermance: oneuser: user1

nix-repl> nixosConfigurations.twousers.config.environment.persistence."/persistence".users.user1.files
trace: impermance: twousers: user1
trace: persistClassHost: twousers
trace: persistClassUser: twousers: user1
trace: impermance: twousers: user2
trace: persistClassHost: twousers
trace: persistClassUser: twousers: user2
trace: impermance: twousers: user1
trace: impermance: twousers: user2

No idea if that is connected to the problem though or normal.

[amanako]

Sorry to have kept you waiting, was busy and tired.
I have to thank you because you helped me understand how my current setup if fragile. I have to admit I was a bit obsessed with keeping it all in one file, while really splitting options and aspect construction was the way to go.
After attempting to solve your exact issue I just couldn't manage to reproduce it, so thought of bettering mine instead. Sorry if this doesn't fit your exact needs but I thought of things which you didn't like about my setup and tried to solve those so hopefully this is what you are looking for. After looking at all those lines again I have to say I was a little disgusted at my current way of doing things. So I will present some changes which if copied exactly should fix the issue. If you have some more specific needs it would be helpful if you point them out.
I thought of the following:

• You didn't like unconditional include in den.schema.host and I didn't like that either when looking at it again. Also it breaks when adding new host so I used lib.optionalAttrs similar to you but instead applied it to whole aspect since that means:
  
  host doesn't enable impermanence -> lib.optionalAttrs returns empty set -> nothing happens to host
  

• I added null to the option instead of default since it doesn't make sense to have some path set if impermanence isn't enabled though I will leave that choice to you.

1. File defining options:

{lib, ...}: {
  den.schema.host = {host, ...}: {
    options.impermanence = lib.mkOption {
      type = lib.types.submodule {
        options.enable = lib.mkOption {
          type = lib.types.bool;
          default = host.impermanence.persistenceDir != null;
          description = "Whether to enable impermanence module";
        };

        options.persistenceDir = lib.mkOption {
          type = lib.types.nullOr lib.types.str;
          default = null;
          description = "Directory for impermanence persistent storage";
        };
      };
    };
  };
}

2. File to construct aspect

{
  den,
  inputs,
  lib,
  ...
}: let
  persistHostClass = {host}:
    den.batteries.forward {
      each = lib.singleton true;
      fromClass = _: "persys";
      intoClass = _: "nixos";
      intoPath = _: ["environment" "persistence" host.impermanence.persistenceDir];
      adaptArgs = {config, ...}: {osConfig = config;};
      guard = {options, ...}: options ? environment.persistence;
    };

  persistUserClass = {
    host,
    user,
  }:
    den.batteries.forward {
      each = lib.singleton true;
      fromClass = _: "persysUser";
      intoClass = _: "nixos";
      intoPath = _: ["environment" "persistence" host.impermanence.persistenceDir "users" user.userName];
      adaptArgs = {config, ...}: {osConfig = config;};
      guard = {options, ...}: options ? environment.persistence;
    };
in {
  flake.den = den;
  flake-file.inputs.impermanence.url = "github:nix-community/impermanence";

  den.aspects.impermanence = {host, ...}:
    lib.optionalAttrs host.impermanence.enable {
      includes = [persistHostClass persistUserClass];
      nixos = {
        imports = [inputs.impermanence.nixosModules.impermanence];
        # This should be minimal for a truly pure setup
        fileSystems."${host.impermanence.persistenceDir}".neededForBoot = true;
      };
    };

  den.schema.host.includes = [den.aspects.impermanence];
}

Overall, there is no { config, ... } which I don't think does what it's supposed to do (if I am not mistaken it represents host options but is quite limited and likely breaks).
There is no deep, hard-to-follow nesting and imports is optional which is important.

Regarding your question I believe classes SHOULD be "scattered around" because that is how aspect communicates with target destination and they get merged in the end anyways.

One caveat I noticed is provides.to-hosts so if you use it to provide configuration for impermanent setup it won't work on systems with multiple users since logically duplication will occur. This case is probably place to use policies but I don't know them too much really.
I tested with your examples though, first 2 are basic and last one is parametric but they all succeeded.

[tschan]

I have at least found a minimal reproducer I think:

# den.nix
{ inputs, den, ... }:
{
  imports = [ inputs.den.flakeModule ];

  # enable strict mode
  den.schema.conf = den.lib.strict;
}

# host.nix
{
  den,
  inputs,
  lib,
  ...
}:
{
  den.hosts.x86_64-linux.host = {
    persistenceDir = "/persistence";

    users = {
      user1 = { };
      user2 = { };
    };
  };

  den.aspects.host = {
    includes = [ den.aspects.impermanence ];
    persist = {
      directories = [
        "/test"
      ];
    };
    nixos = {
      nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
    };
  };
}

# flake.nix
{
  description = "Your new nix config";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    import-tree.url = "github:denful/import-tree";
    den.url = "github:denful/den";
    impermanence.url = "github:nix-community/impermanence";
    impermanence.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs =
    {
      import-tree,
      nixpkgs,
      ...
    }@inputs:
    let
      den =
        (nixpkgs.lib.evalModules {
          modules = [ (import-tree ./den-modules) ];
          specialArgs = {
            inputs = inputs;
          };
        }).config;
    in
    {
      inherit (den.flake) nixosConfigurations darwinConfigurations;
    };
}

With the following impermanence aspect I get an error:

# impermanence.nix
{
  inputs,
  den,
  lib,
  ...
}:
let
  persistClassHost =
    { host }: builtins.trace "persistClassHost: ${host.hostName}"
    den.batteries.forward {
      each = lib.singleton true;
      fromClass = _: "persist";
      intoClass = _: "nixos";
      intoPath = _: [
        "environment"
        "persistence"
        host.persistenceDir
      ];
      adaptArgs =
        { config, ... }:
        {
          osConfig = config;
        };
      guard = { options, ... }: host.persistenceDir != "";
    };
in
{
  den =
    { config, ... }:
    {
      schema.host.options.persistenceDir = lib.mkOption {
        type = lib.types.str;
        default = "";
        description = "Directory for impermanence persistent storage (enables Impermanence for host if set)";
      };

      aspects.impermanence =
        ({ host, user }: builtins.trace "impermance: ${host.hostName}: ${user.userName}"
        (lib.optionalAttrs (host.persistenceDir != "") {
          includes = [
            persistClassHost
          ];

          nixos = {
            imports = [ inputs.impermanence.nixosModules.impermanence ];
          };
        }));
    };
}

nix-repl> nixosConfigurations.host.config.environment.persistence."/persistence".users.user1.files
trace: impermance: host: user1
trace: persistClassHost: host
trace: impermance: host: user2
trace: persistClassHost: host
error:
       … while evaluating a branch condition
         at /nix/store/3a2vdn5i7vd2wl654xs8nb52jf1v6cbh-source/lib/attrsets.nix:1188:11:
         1187|           name: value:
         1188|           if isAttrs value && cond value then recurse (path ++ [ name ]) value else f (path ++ [ name ]) value
             |           ^
         1189|         );

       … in the left operand of the AND (&&) operator
         at /nix/store/3a2vdn5i7vd2wl654xs8nb52jf1v6cbh-source/lib/attrsets.nix:1188:28:
         1187|           name: value:
         1188|           if isAttrs value && cond value then recurse (path ++ [ name ]) value else f (path ++ [ name ]) value
             |                            ^
         1189|         );

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: The option `environment.persistence' in `nixos@impermanence/<anon>:0' is already declared in `nixos@impermanence/<anon>:0'.

With this one it behaves as expected:

{
  inputs,
  den,
  lib,
  ...
}:
let
  persistClassHost =
    { host }: builtins.trace "persistClassHost: ${host.hostName}"
    den.batteries.forward {
      each = lib.singleton true;
      fromClass = _: "persist";
      intoClass = _: "nixos";
      intoPath = _: [
        "environment"
        "persistence"
        host.persistenceDir
      ];
      adaptArgs =
        { config, ... }:
        {
          osConfig = config;
        };
      guard = { options, ... }: host.persistenceDir != "";
    };
in
{
  den =
    { config, ... }:
    {
      schema.host.options.persistenceDir = lib.mkOption {
        type = lib.types.str;
        default = "";
        description = "Directory for impermanence persistent storage (enables Impermanence for host if set)";
      };

      aspects.impermanence =
        ({ host }: builtins.trace "impermance: ${host.hostName}"
        (lib.optionalAttrs (host.persistenceDir != "") {
          includes = [
            persistClassHost
          ];

          nixos = {
            imports = [ inputs.impermanence.nixosModules.impermanence ];
          };
        }));
    };
}

nix-repl> nixosConfigurations.host.config.environment.persistence."/persistence".users.user1.files
trace: impermance: host
trace: persistClassHost: host
error: attribute 'user1' missing
       at «string»:1:1:
            1| nixosConfigurations.host.config.environment.persistence."/persistence".users.user1.files
             | ^
nix-repl> nixosConfigurations.host.config.environment.persistence."/persistence".directories       
[
  { ... }
]

So { host, user } vs. { host } makes the difference. I would like to set "${user.userName}".hashedPasswordFile in the impermanence aspect to load the password hash from a file if Impermanence is enabled. That's why I would like to access the user entity.

[sini]

I actually don't use policies at all for impermanence -- no need for forward rules. I use quirks.

https://github.com/sini/nix-config/blob/main/modules/den/quirks/impermanence.nix
+
https://github.com/sini/nix-config/tree/main/modules/den/aspects/core/impermanence

[fmway]

My question is if this is actually supposed to work like that or if it is a bug?

I don’t understand why that happened either. Seems like a bug. But mine works properly, even in multiple hosts and users. Can you show the full code that related to persist quirks? Maybe i will try it in my configuration to confirm that.

This removes the duplicated but also excludes the persistUservalues from the base preset:

😭

[tschan]

I don’t understand why that happened either. Seems like a bug. But mine works properly, even in multiple hosts and users. Can you show the full code that related to persist quirks? Maybe i will try it in my configuration to confirm that.

My full config is not in a state to be shown because its a bit of a Frankenstein experiment of migrating to den. But I tried to reduce it to the relevant moving parts. In this comment you can see the host aspect and preset.

Here are the quirk related files.

Thanks for taking a look. :)

[fmway]

Here are the quirk related files.

Thanks for taking a look. :)

Okay. I've a few assumptions. It will take a long time. Don't wait for me :)

[fmway]

I got it. The thing is, you apply the impermanence aspect in the sys-secret-basic aspect that will included in to-users. It'll executed twice because you have 2 users. Even you're using lib.unique, this will still duplicated because the mergePersist is called twice.

That's my bad, I just understood the trace you did.

I tried several ways and find one that works. Here for the logic:

1. apply the impermanence aspect in the host aspect instead of to-users policy.
2. expose persist quirk in the user aspect. so the persist quirk in the user aspect can be collected in the host aspect.
3. use policy.include to collecting persistUser quirk for each users.

{ den, lib, ... }: let
  inherit (den.lib) policy;
  inherit (policy) pipe;
in {
  den.aspects.impermanence = {
    persist-collector =
      { host, ... }:
      {
        nixos =
          { persist, lib, ... }: let
            mergePersist = entries: {
              directories = lib.unique (lib.concatMap (e: e.directories or []) entries);
              files = lib.unique (lib.concatMap (e: e.files or []) entries);
            };
          in {
            environment.persistence.${host.persistenceDir} = mergePersist persist;
          };
      };
    persist-user-collector =
      { host, user, ... }:
      {
        nixos =
          { persistUser, lib, ... }: let
            mergePersist = entries: {
              directories = lib.unique (lib.concatMap (e: e.directories or []) entries);
              files = lib.unique (lib.concatMap (e: e.files or []) entries);
            };
          in {
            environment.persistence.${host.persistenceDir}.users.${user.userName} = mergePersist persistUser;
          };
      };
    policies.persist-user-to-host =
      { host, user, ... }: [
        (pipe.from "persist" [ pipe.expose ])
        (policy.include {
          includes = [ den.aspects.impermanence.persist-user-collector ];
        })
      ];

    includes = [
      ({ host, ... }: {
        nixos.fileSystems.${host.persistenceDir}.neededForBoot = true;
        nixos.users.users = lib.mapAttrs' (_: user: {
          name = user.userName;
          value.hashedPasswordFile = "${host.persistenceDir}/paswords/${user.userName}";
        }) (host.users // { root.userName = "root"; });
      })
      den.aspects.impermanence.persist-collector
      den.aspects.impermanence.policies.persist-user-to-host
    ];
  };

  den.aspects.host1 = {
    includes = [
      den.aspects.impermanence
    ];
    provides.to-users =
      { user, ... }:
      {
        includes = [
          den.aspects.sys-secret-basic
        ];
        persistUser = lib.optionalAttrs (user.userName == "user1") {
          directories = [ ".config/git" ];
        };
      };
  };
}

CC: @tschan

[tschan]

Wow, thank you very much! That seems to work as expected.

Just so that I understand it correctly: Any aspect we include in the to-users policy of a host also gets applied to the host itself and that is the intended behaviour?

[fmway]

Just so that I understand it correctly: Any aspect we include in the to-users policy of a host also gets applied to the host itself and that is the intended behaviour?

Yes.

Sorry, maybe i just spammed the notifications. Sometimes i often misunderstand a conversation, especially in English :)

[fmway]

I did something similar to what they did above.

First, I used a forwarder, then I patched the impermanence module by adding deduplication logic. (See this).

The second, I tried an experiment using quirks similar to the one above. The difference is that in the user schema, I set the pipe.expose policy (see this). This is intended to expose the persistence quirk to the parent aspect (host). So I only need to collect the persistence quirk in the host aspect.