nixos-rebuild ready image

[WizardUli]

Hello. I'm trying to create an image (currently for Raspberry Pi 3) where its configuration is copied to it and one can use nixos-rebuild out of the box.
Here is my setup https://gitlab.com/grawp/nixos-raspberry-experiment

Image-building-only related configuration should be inside sd-aarch64-custom.nix while general configuration (which may also be used during image building) should be in nixos/configuration.nix. But I failed somewhere because when I boot a freshly built image on a Raspberry and then run nixos-rebuild switch without any configuration changes, it still tries to download and rebuild derivations galore (and it works OK but that's not the point).

Here I explicitly disconnected internet from the raspberry before executing the nixos-rebuild switch to better visualize what it tried downloading: https://gist.github.com/Grawp/daeb4106a3d9086a70e3c5608c83c782

I want to get to a state where a nixos-generators built system is the same system as the one after running nixos-rebuild switch. In other word I want a system where running first nixos-rebuild switch without changing configuration produce 0 changes.
Any ideas what can be the problem in my setup?

[tv42]

I would assume that the generated image contains what's needed at runtime, but nixos-rebuild also needs build-time things. Looking at what is being downloaded should be informative.

[WizardUli]

@tv42 I've came to the same conclusion and will investigate it further but I think I can close this issue now. Thanks.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-generate-a-nixos-tarball-lxc-template-with-all-dependencies-not-just-runtime/25444/1

[WizardUli]

Btw. I figured it out long time ago and created this module:

({ config, ... }: { sdImage.storePaths = [ nixpkgs config.system.build.toplevel.drvPath ]; })

Simple, isn't it? :D

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-generate-a-nixos-tarball-lxc-template-with-all-dependencies-not-just-runtime/25444/3

[smacz42]

@WizardUli can you explain where you put that module?

EDIT: My problem right now is that ANY nixos-rebuild borks the running container, and it has to be shutdown from the hypervisor:

building '/nix/store/sqz8q13v1y8xn94sx13akafkswaqjsz1-nixos-system-nixos-23.11.5353.878ef7d9721b.drv'...                                                                                                                                      
'/nix/store/sfxgs1pbglvqlxmqnrkijfy9n72qgmbh-system-path/bin/busctl --json=short call org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager ListUnitsByPatterns asas 0 0' exited with value 1 at /nix/store/x5z0
d43d6lhrzkgmaivygbig78lydqcn-nixos-system-nixos-23.11.5353.878ef7d9721b/bin/switch-to-configuration line 145.                                                                                                                                 
warning: error(s) occurred while switching to the new configuration                                                                                                                                                                           
[root@test-cache:~]# shutdown -r now                                                                                                                                                                                                          
Failed to set wall message, ignoring: Access denied                                                                                                                                                                                           
Call to Reboot failed: Access denied                                                                                                                                                                                                          

After a reboot, the configuration works fine. I'm assuming that something that it's pulling in is breaking the system.

When I try to pass it in with github:nix-community/nixos-generators -- --format proxmox-lxc -c /tmp/firstboot.nix as the only entry in firstboot.nix, I get

error:
       … while calling the 'seq' builtin

         at /nix/store/k4jq2035a1sqlqn3g2ici2z43b2yp051-nixos-23.11/nixos/lib/modules.nix:320:18:

          319|         options = checked options;
          320|         config = checked (removeAttrs config [ "_module" ]);
             |                  ^
          321|         _module = checked (config._module);

       … while evaluating a branch condition

         at /nix/store/k4jq2035a1sqlqn3g2ici2z43b2yp051-nixos-23.11/nixos/lib/modules.nix:261:9:

          260|       checkUnmatched =
          261|         if config._module.check && config._module.freeformType == null && merged.unmatchedDefns != [] then
             |         ^
          262|           let

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: undefined variable 'nixpkgs'

       at /tmp/firstboot.nix:8:26:

            7|   # Set the storePaths to include the build top-level derevation path
            8|   sdImage.storePaths = [ nixpkgs config.system.build.toplevel.drvPath ];
             |                          ^
            9|

I'm lacking the understanding of nix to know where it's supposed to go :(

[WizardUli]

@WizardUli can you explain where you put that module?

EDIT: My problem right now is that ANY nixos-rebuild borks the running container, and it has to be shutdown from the hypervisor:

building '/nix/store/sqz8q13v1y8xn94sx13akafkswaqjsz1-nixos-system-nixos-23.11.5353.878ef7d9721b.drv'...                                                                                                                                      
'/nix/store/sfxgs1pbglvqlxmqnrkijfy9n72qgmbh-system-path/bin/busctl --json=short call org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager ListUnitsByPatterns asas 0 0' exited with value 1 at /nix/store/x5z0
d43d6lhrzkgmaivygbig78lydqcn-nixos-system-nixos-23.11.5353.878ef7d9721b/bin/switch-to-configuration line 145.                                                                                                                                 
warning: error(s) occurred while switching to the new configuration                                                                                                                                                                           
[root@test-cache:~]# shutdown -r now                                                                                                                                                                                                          
Failed to set wall message, ignoring: Access denied                                                                                                                                                                                           
Call to Reboot failed: Access denied                                                                                                                                                                                                          

After a reboot, the configuration works fine. I'm assuming that something that it's pulling in is breaking the system.

When I try to pass it in with github:nix-community/nixos-generators -- --format proxmox-lxc -c /tmp/firstboot.nix as the only entry in firstboot.nix, I get

error:
       … while calling the 'seq' builtin

         at /nix/store/k4jq2035a1sqlqn3g2ici2z43b2yp051-nixos-23.11/nixos/lib/modules.nix:320:18:

          319|         options = checked options;
          320|         config = checked (removeAttrs config [ "_module" ]);
             |                  ^
          321|         _module = checked (config._module);

       … while evaluating a branch condition

         at /nix/store/k4jq2035a1sqlqn3g2ici2z43b2yp051-nixos-23.11/nixos/lib/modules.nix:261:9:

          260|       checkUnmatched =
          261|         if config._module.check && config._module.freeformType == null && merged.unmatchedDefns != [] then
             |         ^
          262|           let

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: undefined variable 'nixpkgs'

       at /tmp/firstboot.nix:8:26:

            7|   # Set the storePaths to include the build top-level derevation path
            8|   sdImage.storePaths = [ nixpkgs config.system.build.toplevel.drvPath ];
             |                          ^
            9|

I'm lacking the understanding of nix to know where it's supposed to go :(

I had not been using nixos-generators anymore (confusing wrapper for an already needlesly complicated image builders from nixpkgs) when I wrote that but I used that as a standard module for nixpkgs. I understand that it may be confusing for a newcomer. E.g. it took me some weeks to understand that those modules everybody was talking about are a concept from nixpkgs framework and are processed by nixpkgs and are are not defined by Nix or Nix-language in any way and can be functions as well as sets...etc..

I'll refresh my code, get it more presentable and link it here sometime next week.

[WizardUli]

@smacz42

@WizardUli can you explain where you put that module?

Sorry for very late answer. You should rather use an already made option for this: https://search.nixos.org/options?channel=23.11&show=system.includeBuildDependencies&from=0&size=50&sort=relevance&type=packages&query=includeBuildDependencies