iso: avoid systemd service startup #202

blaggacao · 2021-03-18T22:44:38Z

fixes #194
alternative to #197

Manual Tests

was unrelated

flk install NixOS --impure correctly onto /mnt ❎ (looks like no profile is present)

Issue: #204
Upstream Issue: NixOS/nixpkgs#116938

acceptable build time / closure size ca 850MB (for a simple base OS) ✔️
local profile with cage service is disabled, that is: boots into terminal ✔️
success: air gapped / offline devshell enter ✔️
failure: aire gapped target install: ← non blocking bonus item ❎

$ flk install POS
warning: you don't have internet access; disabling some network-dependent features
building the flake in path:/iso/devos?narHash=sha265-...
warning: you don't have internet access; disabling some network-dependent features
error: unable to download 'https://api.github.com/repos/NixOS/nixpkgs/df8e3...': Couldn't resolve host name (6)

→ detailed rationale in the commit messages

❤️ @Pacman99 for the excellent and detailed discussions in #197 and the may ideas, suggestions and code.

lib/devos/devosSystem.nix

This is required so that filtering via lib.remove works against modules.core and similar which are of path type. It is also a prerequisite for disabledModules to match by module.key instead of path string relative to nixpkgs' modulePath.

IN order to avoid random startup of systemd services, filter out all profiles, except for core and user profiles. This works becasue of a fundamental devos contract, that modules only define configuration, but don't implement them and profiles only implement confguration but don't define them. So only ever an activated profile is expected to effectively start up a systemd service. closes: divnix#194

This ensures that all builds of activated profiles are included into the iso cache and don't require rebuilding within the live installer environment.

This is just for convenience, since the closuers are already in the store. It might be helpful to be able to test out some things of those deactivated profiles een on the iso isntaller.

lib/devos/devosSystem.nix

Pacman99 · 2021-03-19T05:21:27Z

Also apparently getting a segmentation faullt while building the iso is a known issue with nix that happens sometimes. A workaround is to preface the build command with GC_DONT_GC=1

bors · 2021-03-19T12:04:46Z

try

Already running a review

Pacman99

Looks great

Pacman99 · 2021-03-19T16:35:12Z

failure: aire gapped target install: ← non blocking bonus item

Just wondering, is this because of not importing drvPath? Perhaps you could add another build for an offline install iso vs a network install iso. But as you said thats a bonus

blaggacao · 2021-03-19T16:49:42Z

Just wondering, is this because of not importing drvPath?

If I remember correctly, it was trying to download nixpkgs source. — which made me wonder about the registry pinning. But I think to fully understand, we would have to dig through https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/installer/tools/nixos-install.sh

Maybe we can open an issue for air-gapped install, and try to make it work over time.

Pacman99 · 2021-03-19T21:21:06Z

I think it might try and download the registry anyways and perhaps try to update flake. I think theres a --offline flag that you could try and see how far you get.

nrdxp · 2021-03-20T04:16:22Z

lib/devos/mkProfileAttrs.nix

@@ -27,7 +27,7 @@ let mkProfileAttrs =
    f = n: _:
      lib.optionalAttrs
        (lib.pathExists "${dir}/${n}/default.nix")
-        { default = "${dir}/${n}"; }
+        { default = /. + "${dir}/${n}"; }


What's the motivation here?

for the expression in devosSystem lib.remove modules.core suite.allProfiles. For the matching to work all items in the list must be the same type. And in general its better to pass around modules as paths rather than strings.

But also.most importantly for disabledModules to filter.on module.key proper instead of a rleative to.modulePath string.

nrdxp · 2021-03-20T04:37:02Z

lib/devos/devosSystem.nix

@@ -6,27 +6,42 @@ lib.nixosSystem (args // {
    let
      moduleList = builtins.attrValues modules;
      modpath = "nixos/modules";
-      cd = "installer/cd-dvd/installation-cd-minimal-new-kernel.nix";
+
+      fullHostConfig = (lib.nixosSystem (args // { modules = moduleList; })).config;

      isoConfig = (lib.nixosSystem


I would really love to see the bonus goal of your PR implemented. Is it possible that we could create an isoConfig' that is everything that the current isoConfig defines with storeContents abstracted out into a separate variable, while the new isoConfig would just be something like:

{ isoConfig = isoConfig' // { isoImage.storeContents = storeContents ++ [ isoConfig' ]; }

I'm trying to think of a way to do this without triggering infinite recursion, although perhaps I haven't thought this through enough and the above does cause and infinite loop. I'll have more time to test tomorrow and see if I can work it out.

Within the devshell of a live iso, it is ultimately upstream nixos-install at this line that is hindering us, specifically: nix flake metadata evaluates very greedily. Couldn't we obtain that .url — which seemed to be a store path — more cheaply?

On the other hand, the build itself is already fully cached (auto?trusted=1) courtesy of the fullHostConfig's toplevel. That extends to the system profile, which is loaded locally from the build's output

After that there is no potential network call left. So the culprit must be indeed nix flake metadata...

If it is only nixos-install that's blocking us (and it has other problems with flakes), perhaps we could just do what's mentioned in that comment, or something similar to work around it. I personally had to do a legacy nix-build via compat dir to install onto my laptop. Something about my configuration triggered the error mentioned in that thread.

We just moved to a new appartment and, for a varying operational reasons, I probably will be unavailable to follow trough on this bonus item any time soon. (No inet, no setup, obly cell-phone, life getting in the way, ...)

Can we carve this out into an issue while moving on with this PR in its current state?

Sure. I'd like to have a crack at it first though, if you don't mind, as I think it would be pretty straight-forward. If I get it working, I'll pushing a PR to your branch.

I suspect the --offline option would get us quite some mileage (combined with a small patch to nixos-install / flakes-first reconception of it), unfortunately I havn't been able to test this in time.

seems I don't have as much time as I'd hoped. I'll go ahead and merge this and we will try to work it out later.
bors r+

bors · 2021-03-22T22:01:21Z

Build succeeded:

bors · 2021-03-23T02:18:21Z

try

Merge conflict.

This was referenced Mar 18, 2021

iso: only include the host's dependencies #197

Closed

suites: simplify code #201

Closed

Pacman99 reviewed Mar 18, 2021

View reviewed changes

lib/devos/devosSystem.nix Outdated Show resolved Hide resolved

blaggacao force-pushed the da/fix-194 branch 4 times, most recently from 12cfb76 to adcd7f0 Compare March 18, 2021 23:46

This comment has been minimized.

Sign in to view

blaggacao force-pushed the da/fix-194 branch 5 times, most recently from 13af8dd to 54654c0 Compare March 19, 2021 04:27

David Arnold added 5 commits March 18, 2021 23:46

lib: ensure path type in mkProfileAttrs

db716d1

This is required so that filtering via lib.remove works against modules.core and similar which are of path type. It is also a prerequisite for disabledModules to match by module.key instead of path string relative to nixpkgs' modulePath.

style / organization

4358478

iso: add back complete host profile to the nix store

0074476

This ensures that all builds of activated profiles are included into the iso cache and don't require rebuilding within the live installer environment.

iso: ensure tools of deactivated profiles are still available

49b1ad8

This is just for convenience, since the closuers are already in the store. It might be helpful to be able to test out some things of those deactivated profiles een on the iso isntaller.

blaggacao force-pushed the da/fix-194 branch from 54654c0 to 49b1ad8 Compare March 19, 2021 04:46

blaggacao marked this pull request as ready for review March 19, 2021 04:47

blaggacao requested a review from Pacman99 March 19, 2021 04:48

This comment has been minimized.

Sign in to view

Pacman99 reviewed Mar 19, 2021

View reviewed changes

lib/devos/devosSystem.nix Show resolved Hide resolved

This comment has been minimized.

Sign in to view

blaggacao marked this pull request as draft March 19, 2021 14:26

This comment has been minimized.

Sign in to view

blaggacao marked this pull request as ready for review March 19, 2021 16:22

Pacman99 approved these changes Mar 19, 2021

View reviewed changes

nrdxp reviewed Mar 20, 2021

View reviewed changes

bors bot merged commit 870897a into divnix:core Mar 22, 2021

Pacman99 mentioned this pull request Mar 22, 2021

Fully air gapped install iso #207

Open

blaggacao deleted the da/fix-194 branch April 22, 2021 00:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

iso: avoid systemd service startup #202

iso: avoid systemd service startup #202

blaggacao commented Mar 18, 2021 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

Pacman99 commented Mar 19, 2021

This comment has been minimized.

bors bot commented Mar 19, 2021

This comment has been minimized.

This comment has been minimized.

Pacman99 left a comment

Pacman99 commented Mar 19, 2021

blaggacao commented Mar 19, 2021

Pacman99 commented Mar 19, 2021

nrdxp Mar 20, 2021

Pacman99 Mar 20, 2021

blaggacao Mar 20, 2021

nrdxp Mar 20, 2021

blaggacao Mar 20, 2021 •

edited

Loading

nrdxp Mar 20, 2021 •

edited

Loading

blaggacao Mar 21, 2021

nrdxp Mar 21, 2021

blaggacao Mar 22, 2021

nrdxp Mar 22, 2021

bors bot commented Mar 22, 2021

bors bot commented Mar 23, 2021

iso: avoid systemd service startup #202

iso: avoid systemd service startup #202

Conversation

blaggacao commented Mar 18, 2021 • edited Loading

Manual Tests

This comment has been minimized.

This comment has been minimized.

Pacman99 commented Mar 19, 2021

This comment has been minimized.

bors bot commented Mar 19, 2021

try

This comment has been minimized.

This comment has been minimized.

Pacman99 left a comment

Choose a reason for hiding this comment

Pacman99 commented Mar 19, 2021

blaggacao commented Mar 19, 2021

Pacman99 commented Mar 19, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blaggacao Mar 20, 2021 • edited Loading

Choose a reason for hiding this comment

nrdxp Mar 20, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bors bot commented Mar 22, 2021

bors bot commented Mar 23, 2021

try

blaggacao commented Mar 18, 2021 •

edited

Loading

blaggacao Mar 20, 2021 •

edited

Loading

nrdxp Mar 20, 2021 •

edited

Loading