blogs.xml

<?xml version="1.0"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">

<channel>
	<title>NixOS Planet</title>
	<link>https://planet.nixos.org</link>
	<language>en</language>
	<description>NixOS Planet - https://planet.nixos.org</description>
	<atom:link rel="self" href="https://planet.nixos.org/rss20.xml" type="application/rss+xml"/>

<item>
	<title>Craige McWhirter: Building Daedalus Flight on NixOS</title>
	<guid isPermaLink="true">http://mcwhirter.com.au//craige/blog/2020/Building_Daedalus_Flight_on_NixOS/</guid>
	<link>http://mcwhirter.com.au//craige/blog/2020/Building_Daedalus_Flight_on_NixOS/</link>
	<description>&lt;p&gt;&lt;img alt=&quot;NixOS Daedalus Gears by Craige McWhirter&quot; src=&quot;http://mcwhirter.com.au/files/NixOS_Daedalus_Gears.png&quot; title=&quot;NixOS Daedalus Gears by Craige McWhirter&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://daedaluswallet.io/en/flight/&quot;&gt;Daedalus Flight&lt;/a&gt; was recently released
and this is how you can build and run this version of
&lt;a href=&quot;https://daedaluswallet.io/&quot;&gt;Deadalus&lt;/a&gt; on &lt;a href=&quot;https://nixos.org/&quot;&gt;NixOS&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If you want to speed the build process up, you can add the
&lt;a href=&quot;https://iohk.io/&quot;&gt;IOHK&lt;/a&gt; &lt;a href=&quot;https://nixos.org/nix/&quot;&gt;Nix&lt;/a&gt; cache to your own NixOS configuration:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://source.mcwhirter.io/craige/mio-ops/src/branch/master/roles/iohk.nix&quot;&gt;iohk.nix&lt;/a&gt;:&lt;/p&gt;

&lt;pre&gt;&lt;code class=&quot;nix&quot;&gt;nix.binaryCaches = [
  &quot;https://cache.nixos.org&quot;
  &quot;https://hydra.iohk.io&quot;
];
nix.binaryCachePublicKeys = [
  &quot;hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ=&quot;
];
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;If you haven't already, you can clone the &lt;a href=&quot;https://github.com/input-output-hk/daedalus&quot;&gt;Daedalus
repo&lt;/a&gt; and specifically the
1.0.0 tagged commit:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;$ git clone --branch 1.0.0 https://github.com/input-output-hk/daedalus.git
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Once you've cloned the repo and checked you're on the 1.0.0 tagged commit,
you can build Daedalus flight with the following command:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;$ nix build -f . daedalus --argstr cluster mainnet_flight
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Once the build completes, you're ready to launch Daedalus Flight:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;$ ./result/bin/daedalus
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;To verify that you have in fact built Daedalus Flight, first head to the
&lt;code&gt;Daedalus&lt;/code&gt; menu then &lt;code&gt;About Daedalus&lt;/code&gt;. You should see a title such as
&quot;DAEDALUS 1.0.0&quot;. The second check, is to press &lt;code&gt;[Ctl]+d&lt;/code&gt; to access &lt;code&gt;Daedalus
Diagnostocs&lt;/code&gt; and your &lt;code&gt;Daedalus state directory&lt;/code&gt; should have &lt;code&gt;mainnet_flight&lt;/code&gt;
at the end of the path.&lt;/p&gt;

&lt;p&gt;If you've got these, give yourself a pat on the back and grab yourself a
refreshing bevvy while you wait for blocks to sync.&lt;/p&gt;

&lt;p&gt;&lt;img alt=&quot;Daedalus FC1 screenshot&quot; src=&quot;http://mcwhirter.com.au/files/Daedalus_FC1.png&quot; title=&quot;Daedalus FC1 screenshot&quot; /&gt;&lt;/p&gt;</description>
	<pubDate>Thu, 23 Apr 2020 23:28:59 +0000</pubDate>
</item>
<item>
	<title>nixbuild.net: Binary Cache Support</title>
	<guid isPermaLink="true">https://blog.nixbuild.net/posts/2020-04-18-binary-cache-support.html</guid>
	<link>https://blog.nixbuild.net/posts/2020-04-18-binary-cache-support.html</link>
	<description>&lt;p&gt;Up until now, nixbuild.net has not supported directly fetching build dependencies from binary caches like &lt;a href=&quot;https://cache.nixos.org&quot;&gt;cache.nixos.org&lt;/a&gt; or &lt;a href=&quot;https://cachix.org&quot;&gt;Cachix&lt;/a&gt;. All build dependencies have instead been uploaded from the user’s local machine to nixbuild.net the first time they’ve been needed.&lt;/p&gt;
&lt;p&gt;Today, this bottleneck has been removed, since nixbuild.net now can fetch build dependencies directly from binary caches, without taxing users’ upload bandwidth.&lt;/p&gt;

&lt;p&gt;By default, the official Nix binary cache (&lt;a href=&quot;https://cache.nixos.org&quot;&gt;cache.nixos.org&lt;/a&gt;) is added to all nixbuild.net accounts, but a nixbuild.net user can freely decide on which caches that should be queried for build dependencies (including &lt;a href=&quot;https://cachix.org&quot;&gt;Cachix&lt;/a&gt; caches).&lt;/p&gt;
&lt;p&gt;An additional benefit of the new support for binary caches is that users that trust the same binary caches automatically share build dependencies from those caches. This means that if one user’s build has triggered a download from for example cache.nixos.org, the next user that comes along and needs the same build dependency doesn’t have to spend time on downloading that dependency.&lt;/p&gt;
&lt;p&gt;For more information on how to use binary caches with nixbuild.net, see the &lt;a href=&quot;https://docs.nixbuild.net/getting-started/&quot;&gt;documentation&lt;/a&gt;.&lt;/p&gt;</description>
	<pubDate>Sat, 18 Apr 2020 00:00:00 +0000</pubDate>
	<author>support@nixbuild.net (nixbuild.net)</author>
</item>
<item>
	<title>Graham Christensen: Erase your darlings</title>
	<guid isPermaLink="false">http://grahamc.com//blog/erase-your-darlings</guid>
	<link>http://grahamc.com/blog/erase-your-darlings</link>
	<description>&lt;p&gt;I erase my systems at every boot.&lt;/p&gt;

&lt;p&gt;Over time, a system collects state on its root partition. This state
lives in assorted directories like &lt;code class=&quot;highlighter-rouge&quot;&gt;/etc&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;/var&lt;/code&gt;, and represents
every under-documented or out-of-order step in bringing up the
services.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;“Right, run &lt;code class=&quot;highlighter-rouge&quot;&gt;myapp-init&lt;/code&gt;.”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;These small, inconsequential “oh, oops” steps are the pieces that get
lost and don’t appear in your runbooks.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;“Just download ca-certificates to … to fix …”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Each of these quick fixes leaves you doomed to repeat history in three
years when you’re finally doing that dreaded RHEL 7 to RHEL 8 upgrade.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;“Oh, &lt;code class=&quot;highlighter-rouge&quot;&gt;touch /etc/ipsec.secrets&lt;/code&gt; or the l2tp tunnel won’t work.”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h3 id=&quot;immutable-infrastructure-gets-us-so-close&quot;&gt;Immutable infrastructure gets us &lt;em&gt;so&lt;/em&gt; close&lt;/h3&gt;

&lt;p&gt;Immutable infrastructure is a wonderfully effective method of
eliminating so many of these forgotten steps. Leaning in to the pain
by deleting and replacing your servers on a weekly or monthly basis
means you are constantly testing and exercising your automation and
runbooks.&lt;/p&gt;

&lt;p&gt;The nugget here is the regular and indiscriminate removal of system
state. Destroying the whole server doesn’t leave you much room to
forget the little tweaks you made along the way.&lt;/p&gt;

&lt;p&gt;These techniques work great when you meet two requirements:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;you can provision and destroy servers with an API call&lt;/li&gt;
  &lt;li&gt;the servers aren’t inherently stateful&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;long-running-servers&quot;&gt;Long running servers&lt;/h4&gt;

&lt;p&gt;There are lots of cases in which immutable infrastructure &lt;em&gt;doesn’t&lt;/em&gt;
work, and the dirty secret is &lt;strong&gt;those servers need good tools the
most.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Long-running servers cause long outages. Their runbooks are outdated
and incomplete. They accrete tweaks and turn in to an ossified,
brittle snowflake — except its arms are load-bearing.&lt;/p&gt;

&lt;p&gt;Let’s bring the ideas of immutable infrastructure to these systems
too. Whether this system is embedded in a stadium’s jumbotron, in a
datacenter, or under your desk, we &lt;em&gt;can&lt;/em&gt; keep the state under control.&lt;/p&gt;

&lt;h4 id=&quot;fhs-isnt-enough&quot;&gt;FHS isn’t enough&lt;/h4&gt;

&lt;p&gt;The hard part about applying immutable techniques to long running
servers is knowing exactly where your application state ends and the
operating system, software, and configuration begin.&lt;/p&gt;

&lt;p&gt;This is hard because legacy operating systems and the Filesystem
Hierarchy Standard poorly separate these areas of concern. For
example, &lt;code class=&quot;highlighter-rouge&quot;&gt;/var/lib&lt;/code&gt; is for state information, but how much of this do
you actually care about tracking? What did you configure in &lt;code class=&quot;highlighter-rouge&quot;&gt;/etc&lt;/code&gt; on
purpose?&lt;/p&gt;

&lt;p&gt;The answer is probably not a lot.&lt;/p&gt;

&lt;p&gt;You may not care, but all of this accumulation of junk is a tarpit.
Everything becomes harder: replicating production, testing changes,
undoing mistakes.&lt;/p&gt;

&lt;h3 id=&quot;new-computer-smell&quot;&gt;New computer smell&lt;/h3&gt;

&lt;p&gt;Getting a new computer is this moment of cleanliness. The keycaps
don’t have oils on them, the screen is perfect, and the hard drive
is fresh and unspoiled — for about an hour or so.&lt;/p&gt;

&lt;p&gt;Let’s get back to that.&lt;/p&gt;

&lt;h2 id=&quot;how-is-this-possible&quot;&gt;How is this possible?&lt;/h2&gt;

&lt;p&gt;NixOS can boot with only two directories: &lt;code class=&quot;highlighter-rouge&quot;&gt;/boot&lt;/code&gt;, and &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;/nix&lt;/code&gt; contains read-only system configurations, which are specified
by your &lt;code class=&quot;highlighter-rouge&quot;&gt;configuration.nix&lt;/code&gt; and are built and tracked as system
generations. These never change. Once the files are created in &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix&lt;/code&gt;,
the only way to change the config’s contents is to build a new system
configuration with the contents you want.&lt;/p&gt;

&lt;p&gt;Any configuration or files created on the drive outside of &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix&lt;/code&gt; is
state and cruft. We can lose everything outside of &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;/boot&lt;/code&gt;
and have a healthy system. My technique is to explicitly opt in and
&lt;em&gt;choose&lt;/em&gt; which state is important, and only keep that.&lt;/p&gt;

&lt;p&gt;How this is possible comes down to the boot sequence.&lt;/p&gt;

&lt;p&gt;For NixOS, the bootloader follows the same basic steps as a standard
Linux distribution: the kernel starts with an initial ramdisk, and the
initial ramdisk mounts the system disks.&lt;/p&gt;

&lt;p&gt;And here is where the similarities end.&lt;/p&gt;

&lt;h3 id=&quot;nixoss-early-startup&quot;&gt;NixOS’s early startup&lt;/h3&gt;

&lt;p&gt;NixOS configures the bootloader to pass some extra information: a
specific system configuration. This is the secret to NixOS’s
bootloader rollbacks, and also the key to erasing our disk on each
boot. The parameter is named &lt;code class=&quot;highlighter-rouge&quot;&gt;systemConfig&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;On every startup the very early boot stage knows what the system’s
configuration should be: the entire system configuration is stored in
the read-only &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix/store&lt;/code&gt;, and the directory passed through
&lt;code class=&quot;highlighter-rouge&quot;&gt;systemConfig&lt;/code&gt; has a reference to the config. Early boot then
manipulates &lt;code class=&quot;highlighter-rouge&quot;&gt;/etc&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;/run&lt;/code&gt; to match the chosen setup. Usually this
involves swapping out a few symlinks.&lt;/p&gt;

&lt;p&gt;If &lt;code class=&quot;highlighter-rouge&quot;&gt;/etc&lt;/code&gt; simply doesn’t exist, however, early boot &lt;em&gt;creates&lt;/em&gt; &lt;code class=&quot;highlighter-rouge&quot;&gt;/etc&lt;/code&gt;
and moves on like it were any other boot. It also &lt;em&gt;creates&lt;/em&gt; &lt;code class=&quot;highlighter-rouge&quot;&gt;/var&lt;/code&gt;,
&lt;code class=&quot;highlighter-rouge&quot;&gt;/dev&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;/home&lt;/code&gt;, and any other core directories that must be present.&lt;/p&gt;

&lt;p&gt;Simply speaking, an empty &lt;code class=&quot;highlighter-rouge&quot;&gt;/&lt;/code&gt; is &lt;em&gt;not surprising&lt;/em&gt; to NixOS. In fact,
the NixOS netboot, EC2, and installation media all start out this way.&lt;/p&gt;

&lt;h2 id=&quot;opting-out&quot;&gt;Opting out&lt;/h2&gt;

&lt;p&gt;Before we can opt in to saving data, we must opt out of saving data
&lt;em&gt;by default&lt;/em&gt;. I do this by setting up my filesystem in a way that
lets me easily and safely erase the unwanted data, while preserving
the data I do want to keep.&lt;/p&gt;

&lt;p&gt;My preferred method for this is using a ZFS dataset and rolling it
back to a blank snapshot before it is mounted. A partition of any
other filesystem would work just as well too, running &lt;code class=&quot;highlighter-rouge&quot;&gt;mkfs&lt;/code&gt; at boot,
or something similar. If you have a lot of RAM, you could skip the
erase step and make &lt;code class=&quot;highlighter-rouge&quot;&gt;/&lt;/code&gt; a tmpfs.&lt;/p&gt;

&lt;h3 id=&quot;opting-out-with-zfs&quot;&gt;Opting out with ZFS&lt;/h3&gt;
&lt;p&gt;When installing NixOS, I partition my disk with two partitions, one
for the boot partition, and another for a ZFS pool. Then I create and
mount a few datasets.&lt;/p&gt;

&lt;p&gt;My root dataset:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# zfs create -p -o mountpoint=legacy rpool/local/root
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Before I even mount it, I &lt;strong&gt;create a snapshot while it is totally
blank&lt;/strong&gt;:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# zfs snapshot rpool/local/root@blank
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And then mount it:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# mount -t zfs rpool/local/root /mnt
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then I mount the partition I created for the &lt;code class=&quot;highlighter-rouge&quot;&gt;/boot&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# mkdir /mnt/boot
# mount /dev/the-boot-partition /mnt/boot
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Create and mount a dataset for &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# zfs create -p -o mountpoint=legacy rpool/local/nix
# mkdir /mnt/nix
# mount -t zfs rpool/local/nix /mnt/nix
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And a dataset for &lt;code class=&quot;highlighter-rouge&quot;&gt;/home&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# zfs create -p -o mountpoint=legacy rpool/safe/home
# mkdir /mnt/home
# mount -t zfs rpool/safe/home /mnt/home
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And finally, a dataset explicitly for state I want to persist between
boots:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# zfs create -p -o mountpoint=legacy rpool/safe/persist
# mkdir /mnt/persist
# mount -t zfs rpool/safe/persist /mnt/persist
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;blockquote&gt;
  &lt;p&gt;&lt;em&gt;Note:&lt;/em&gt; in my systems, datasets under &lt;code class=&quot;highlighter-rouge&quot;&gt;rpool/local&lt;/code&gt; are never backed
up, and datasets under &lt;code class=&quot;highlighter-rouge&quot;&gt;rpool/safe&lt;/code&gt; are.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;And now safely erasing the root dataset on each boot is very easy:
after devices are made available, roll back to the blank snapshot:&lt;/p&gt;

&lt;div class=&quot;language-nix highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;boot&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;initrd&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;postDeviceCommands&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;lib&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;mkAfter&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;''&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;    zfs rollback -r rpool/local/root@blank&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;  ''&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;I then finish the installation as normal. If all goes well, your
next boot will start with an empty root partition but otherwise be
configured exactly as you specified.&lt;/p&gt;

&lt;h2 id=&quot;opting-in&quot;&gt;Opting in&lt;/h2&gt;

&lt;p&gt;Now that I’m keeping no state, it is time to specify what I do want
to keep. My choices here are different based on the role of the
system: a laptop has different state than a server.&lt;/p&gt;

&lt;p&gt;Here are some different pieces of state and how I preserve them. These
examples largely use reconfiguration or symlinks, but using ZFS
datasets and mount points would work too.&lt;/p&gt;

&lt;h4 id=&quot;wireguard-private-keys&quot;&gt;Wireguard private keys&lt;/h4&gt;

&lt;p&gt;Create a directory under &lt;code class=&quot;highlighter-rouge&quot;&gt;/persist&lt;/code&gt; for the key:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# mkdir -p /persist/etc/wireguard/
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And use Nix’s wireguard module to generate the key there:&lt;/p&gt;

&lt;div class=&quot;language-nix highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;networking&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;wireguard&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;interfaces&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;wg0&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;generatePrivateKeyFile&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;privateKeyFile&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;/persist/etc/wireguard/wg0&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;p&quot;&gt;};&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;networkmanager-connections&quot;&gt;NetworkManager connections&lt;/h4&gt;

&lt;p&gt;Create a directory under &lt;code class=&quot;highlighter-rouge&quot;&gt;/persist&lt;/code&gt;, mirroring the &lt;code class=&quot;highlighter-rouge&quot;&gt;/etc&lt;/code&gt; structure:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# mkdir -p /persist/etc/NetworkManager/system-connections
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And use Nix’s &lt;code class=&quot;highlighter-rouge&quot;&gt;etc&lt;/code&gt; module to set up the symlink:&lt;/p&gt;

&lt;div class=&quot;language-nix highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;etc&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;NetworkManager/system-connections&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;source&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;/persist/etc/NetworkManager/system-connections/&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
  &lt;span class=&quot;p&quot;&gt;};&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;bluetooth-devices&quot;&gt;Bluetooth devices&lt;/h4&gt;

&lt;p&gt;Create a directory under &lt;code class=&quot;highlighter-rouge&quot;&gt;/persist&lt;/code&gt;, mirroring the &lt;code class=&quot;highlighter-rouge&quot;&gt;/var&lt;/code&gt; structure:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# mkdir -p /persist/var/lib/bluetooth
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And then use systemd’s tmpfiles.d rules to create a symlink from
&lt;code class=&quot;highlighter-rouge&quot;&gt;/var/lib/bluetooth&lt;/code&gt; to my persisted directory:&lt;/p&gt;

&lt;div class=&quot;language-nix highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;systemd&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;tmpfiles&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;rules&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;
    &lt;span class=&quot;s2&quot;&gt;&quot;L /var/lib/bluetooth - - - - /persist/var/lib/bluetooth&quot;&lt;/span&gt;
  &lt;span class=&quot;p&quot;&gt;];&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;ssh-host-keys&quot;&gt;SSH host keys&lt;/h4&gt;

&lt;p&gt;Create a directory under &lt;code class=&quot;highlighter-rouge&quot;&gt;/persist&lt;/code&gt;, mirroring the &lt;code class=&quot;highlighter-rouge&quot;&gt;/etc&lt;/code&gt; structure:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# mkdir -p /persist/etc/ssh
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And use Nix’s openssh module to create and use the keys in that
directory:&lt;/p&gt;

&lt;div class=&quot;language-nix highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;services&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;openssh&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;enable&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
    &lt;span class=&quot;nv&quot;&gt;hostKeys&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;
      &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;nv&quot;&gt;path&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;/persist/ssh/ssh_host_ed25519_key&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;nv&quot;&gt;type&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;ed25519&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
      &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
      &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
        &lt;span class=&quot;nv&quot;&gt;path&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;/persist/ssh/ssh_host_rsa_key&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;nv&quot;&gt;type&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;rsa&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
        &lt;span class=&quot;nv&quot;&gt;bits&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;4096&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;;&lt;/span&gt;
      &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
    &lt;span class=&quot;p&quot;&gt;];&lt;/span&gt;
  &lt;span class=&quot;p&quot;&gt;};&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h4 id=&quot;acme-certificates&quot;&gt;ACME certificates&lt;/h4&gt;

&lt;p&gt;Create a directory under &lt;code class=&quot;highlighter-rouge&quot;&gt;/persist&lt;/code&gt;, mirroring the &lt;code class=&quot;highlighter-rouge&quot;&gt;/var&lt;/code&gt; structure:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# mkdir -p /persist/var/lib/acme
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And then use systemd’s tmpfiles.d rules to create a symlink from
&lt;code class=&quot;highlighter-rouge&quot;&gt;/var/lib/acme&lt;/code&gt; to my persisted directory:&lt;/p&gt;

&lt;div class=&quot;language-nix highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;systemd&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;tmpfiles&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;rules&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;
    &lt;span class=&quot;s2&quot;&gt;&quot;L /var/lib/acme - - - - /persist/var/lib/acme&quot;&lt;/span&gt;
  &lt;span class=&quot;p&quot;&gt;];&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;answering-the-question-what-am-i-about-to-lose&quot;&gt;Answering the question “what am I about to lose?”&lt;/h3&gt;

&lt;p&gt;I found this process a bit scary for the first few weeks: was I losing
important data each reboot? No, I wasn’t.&lt;/p&gt;

&lt;p&gt;If you’re worried and want to know what state you’ll lose on the next
boot, you can list the files on your root filesystem and see if you’re
missing something important:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# tree -x /
├── bin
│   └── sh -&amp;gt; /nix/store/97zzcs494vn5k2yw-dash-0.5.10.2/bin/dash
├── boot
├── dev
├── etc
│   ├── asound.conf -&amp;gt; /etc/static/asound.conf
... snip ...
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;ZFS can give you a similar answer:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# zfs diff rpool/local/root@blank
M	/
+	/nix
+	/etc
+	/root
+	/var/lib/is-nix-channel-up-to-date
+	/etc/pki/fwupd
+	/etc/pki/fwupd-metadata
... snip ...
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;your-stateless-future&quot;&gt;Your stateless future&lt;/h2&gt;

&lt;p&gt;You may bump in to new state you meant to be preserving. When I’m
adding new services, I think about the state it is writing and whether
I care about it or not. If I care, I find a way to redirect its state
to &lt;code class=&quot;highlighter-rouge&quot;&gt;/persist&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Take care to reboot these machines on a somewhat regular basis. It
will keep things agile, proving your system state is tracked
correctly.&lt;/p&gt;

&lt;p&gt;This technique has given me the “new computer smell” on every boot
without the datacenter full of hardware, and even on systems that do
carry important state. I have deployed this strategy to systems in the
large and small: build farm servers, database servers, my NAS and home
server, my raspberry pi garage door opener, and laptops.&lt;/p&gt;

&lt;p&gt;NixOS enables powerful new deployment models in so many ways, allowing
for systems of all shapes and sizes to be managed properly and
consistently. I think this model of ephemeral roots is yet
another example of this flexibility and power. I would like to see
this partitioning scheme become a reference architecture and take us
out of this eternal tarpit of legacy.&lt;/p&gt;</description>
	<pubDate>Mon, 13 Apr 2020 00:00:00 +0000</pubDate>
</item>
<item>
	<title>Graham Christensen: ZFS Datasets for NixOS</title>
	<guid isPermaLink="false">http://grahamc.com//blog/nixos-on-zfs</guid>
	<link>http://grahamc.com/blog/nixos-on-zfs</link>
	<description>&lt;p&gt;The outdated and historical nature of the &lt;a href=&quot;https://grahamc.com/feed/fhs&quot;&gt;Filesystem Hierarchy
Standard&lt;/a&gt; means traditional Linux distributions have to go to great
lengths to separate “user data” from “system data.”&lt;/p&gt;

&lt;p&gt;NixOS’s filesystem architecture does cleanly separate user data from
system data, and has a much easier job to do.&lt;/p&gt;

&lt;h3 id=&quot;traditional-linuxes&quot;&gt;Traditional Linuxes&lt;/h3&gt;

&lt;p&gt;Because FHS mixes these two concerns across the entire hierarchy,
splitting these concerns requires identifying every point across
dozens of directories where the data is the system’s or the user’s.
When adding ZFS to the mix, the installers typically have to create
over a dozen datasets to accomplish this.&lt;/p&gt;

&lt;p&gt;For example, Ubuntu’s upcoming ZFS support creates 16 datasets:&lt;/p&gt;

&lt;pre&gt;&lt;code class=&quot;language-tree&quot;&gt;rpool/
├── ROOT
│   └── ubuntu_lwmk7c
│       ├── log
│       ├── mail
│       ├── snap
│       ├── spool
│       ├── srv
│       ├── usr
│       │   └── local
│       ├── var
│       │   ├── games
│       │   └── lib
│       │       ├── AccountServices
│       │       ├── apt
│       │       ├── dpkg
│       │       └── NetworkManager
│       └── www
└── USERDATA
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Going through the great pains of separating this data comes with
significant advantages: a recursive snapshot at any point in the tree
will create an atomic, point-in-time snapshot of every dataset below.&lt;/p&gt;

&lt;p&gt;This means in order to create a consistent snapshot of the system
data, an administrator would only need to take a recursive snapshot
at &lt;code class=&quot;highlighter-rouge&quot;&gt;ROOT&lt;/code&gt;. The same is true for user data: take a recursive snapshot of
&lt;code class=&quot;highlighter-rouge&quot;&gt;USERDATA&lt;/code&gt; and all user data is saved.&lt;/p&gt;

&lt;h3 id=&quot;nixos&quot;&gt;NixOS&lt;/h3&gt;

&lt;p&gt;Because Nix stores all of its build products in &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix/store&lt;/code&gt;, NixOS
doesn’t mingle these two concerns. NixOS’s runtime system, installed
packages, and rollback targets are all stored in &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;User data is not.&lt;/p&gt;

&lt;p&gt;This removes the entire complicated tree of datasets to facilitate
FHS, and leaves us with only a few needed datasets.&lt;/p&gt;

&lt;h2 id=&quot;datasets&quot;&gt;Datasets&lt;/h2&gt;

&lt;p&gt;Design for the atomic, recursive snapshots when laying out the
datasets.&lt;/p&gt;

&lt;p&gt;In particular, I don’t back up the &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix&lt;/code&gt; directory. This entire
directory can always be rebuilt later from the system’s
&lt;code class=&quot;highlighter-rouge&quot;&gt;configuration.nix&lt;/code&gt;, and isn’t worth the space.&lt;/p&gt;

&lt;p&gt;One way to model this might be splitting up the data into three
top-level datasets:&lt;/p&gt;

&lt;pre&gt;&lt;code class=&quot;language-tree&quot;&gt;tank/
├── local
│   └── nix
├── system
│   └── root
└── user
    └── home
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;In &lt;code class=&quot;highlighter-rouge&quot;&gt;tank/local&lt;/code&gt;, I would store datasets that should almost never be
snapshotted or backed up. &lt;code class=&quot;highlighter-rouge&quot;&gt;tank/system&lt;/code&gt; would store data that I would
want periodic snapshots for. Most importantly, &lt;code class=&quot;highlighter-rouge&quot;&gt;tank/user&lt;/code&gt; would
contain data I want regular snapshots and backups for, with a long
retention policy.&lt;/p&gt;

&lt;p&gt;From here, you could add a ZFS dataset per user:&lt;/p&gt;

&lt;pre&gt;&lt;code class=&quot;language-tree&quot;&gt;tank/
├── local
│   └── nix
├── system
│   └── root
└── user
    └── home
        ├── grahamc
        └── gustav
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Or a separate dataset for &lt;code class=&quot;highlighter-rouge&quot;&gt;/var&lt;/code&gt;:&lt;/p&gt;

&lt;pre&gt;&lt;code class=&quot;language-tree&quot;&gt;tank/
├── local
│   └── nix
├── system
│   ├── var
│   └── root
└── user
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Importantly, this gives you three buckets for independent and
regular snapshots.&lt;/p&gt;

&lt;p&gt;The important part is having &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix&lt;/code&gt; under its own top-level dataset.
This makes it a “cousin” to the data you &lt;em&gt;do&lt;/em&gt; want backup coverage on,
making it easier to take deep, recursive snapshots atomically.&lt;/p&gt;

&lt;h2 id=&quot;properties&quot;&gt;Properties&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;Enable compression with &lt;code class=&quot;highlighter-rouge&quot;&gt;compression=on&lt;/code&gt;. Specifying &lt;code class=&quot;highlighter-rouge&quot;&gt;on&lt;/code&gt; instead of
&lt;code class=&quot;highlighter-rouge&quot;&gt;lz4&lt;/code&gt; or another specific algorithm will always pick the best
available compression algorithm.&lt;/li&gt;
  &lt;li&gt;The dataset containing journald’s logs (where &lt;code class=&quot;highlighter-rouge&quot;&gt;/var&lt;/code&gt; lives) should
have &lt;code class=&quot;highlighter-rouge&quot;&gt;xattr=sa&lt;/code&gt; and &lt;code class=&quot;highlighter-rouge&quot;&gt;acltype=posixacl&lt;/code&gt; set to allow regular users to
read their journal.&lt;/li&gt;
  &lt;li&gt;Nix doesn’t use &lt;code class=&quot;highlighter-rouge&quot;&gt;atime&lt;/code&gt;, so &lt;code class=&quot;highlighter-rouge&quot;&gt;atime=off&lt;/code&gt; on the &lt;code class=&quot;highlighter-rouge&quot;&gt;/nix&lt;/code&gt; dataset is
fine.&lt;/li&gt;
  &lt;li&gt;NixOS requires (as of 2020-04-11) &lt;code class=&quot;highlighter-rouge&quot;&gt;mountpoint=legacy&lt;/code&gt; for all
datasets. NixOS does not yet have tooling to require implicitly
created ZFS mounts to settle before booting, and &lt;code class=&quot;highlighter-rouge&quot;&gt;mountpoint=legacy&lt;/code&gt;
plus explicit mount points in &lt;code class=&quot;highlighter-rouge&quot;&gt;hardware-configuration.nix&lt;/code&gt; will
ensure all your datasets are mounted at the right time.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I don’t know how to pick &lt;code class=&quot;highlighter-rouge&quot;&gt;ashift&lt;/code&gt;, and usually just allow ZFS to guess
on my behalf.&lt;/p&gt;

&lt;h2 id=&quot;partitioning&quot;&gt;Partitioning&lt;/h2&gt;

&lt;p&gt;I only create two partitions:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;/boot&lt;/code&gt; formatted &lt;code class=&quot;highlighter-rouge&quot;&gt;vfat&lt;/code&gt; for EFI, or &lt;code class=&quot;highlighter-rouge&quot;&gt;ext4&lt;/code&gt; for BIOS&lt;/li&gt;
  &lt;li&gt;The ZFS dataset partition.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;There are spooky articles saying only give ZFS entire disks. The
truth is, you shouldn’t split a disk into two active partitions.
Splitting the disk this way is just fine, since &lt;code class=&quot;highlighter-rouge&quot;&gt;/boot&lt;/code&gt; is rarely
read or written.&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;&lt;em&gt;Note:&lt;/em&gt; If you do partition the disk, make sure you set the disk’s
scheduler to &lt;code class=&quot;highlighter-rouge&quot;&gt;none&lt;/code&gt;. ZFS takes this step automatically if it does
control the entire disk.&lt;/p&gt;

  &lt;p&gt;On NixOS, you an set your scheduler to &lt;code class=&quot;highlighter-rouge&quot;&gt;none&lt;/code&gt; via:&lt;/p&gt;

  &lt;div class=&quot;language-nix highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;boot&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;kernelParams&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;elevator=none&quot;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;];&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;  &lt;/div&gt;
&lt;/blockquote&gt;

&lt;h1 id=&quot;clean-isolation&quot;&gt;Clean isolation&lt;/h1&gt;

&lt;p&gt;NixOS’s clean separation of concerns reduces the amount of complexity
we need to track when considering and planning our datasets. This
gives us flexibility later, and enables some superpowers like erasing
my computer on every boot, which I’ll write about on Monday.&lt;/p&gt;</description>
	<pubDate>Sat, 11 Apr 2020 00:00:00 +0000</pubDate>
</item>
<item>
	<title>nixbuild.net: New nixbuild.net Resources</title>
	<guid isPermaLink="true">https://blog.nixbuild.net/posts/2020-03-27-nixbuild-net-beta.html</guid>
	<link>https://blog.nixbuild.net/posts/2020-03-27-nixbuild-net-beta.html</link>
	<description>&lt;p&gt;On the support side of the nixbuild.net service, two new resources have been published:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://docs.nixbuild.net&quot;&gt;docs.nixbuild.net&lt;/a&gt;, collecting all available documentation for nixbuild.net users.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;The &lt;a href=&quot;https://github.com/nixbuild/feedback&quot;&gt;nixbuild.net feedback&lt;/a&gt; repository on GitHub, providing a way to report issues or ask questions related to the service.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These resources are mainly useful for nixbuild.net beta users, but they are open to anyone. And anyone is of course welcome to request a free beta account for evaluating nixbuild.net, by just &lt;a href=&quot;mailto:rickard@nixbuild.net&quot;&gt;sending me an email&lt;/a&gt;.&lt;/p&gt;</description>
	<pubDate>Fri, 27 Mar 2020 00:00:00 +0000</pubDate>
	<author>support@nixbuild.net (nixbuild.net)</author>
</item>
<item>
	<title>Matthew Bauer: Announcing Nixiosk</title>
	<guid isPermaLink="true">https://matthewbauer.us/blog/nixiosk.html</guid>
	<link>https://matthewbauer.us/blog/nixiosk.html</link>
	<description>&lt;p&gt;
Today I’m announcing a project I’ve been working on for the last few
weeks. I’m calling it Nixiosk which is kind of a smashing together of
the words NixOS and Kiosk. The idea is to have an easy way to make
locked down, declarative systems
&lt;/p&gt;

&lt;p&gt;
My main application of this is my two Raspberry Pi systems that I own.
Quite a few people have installed NixOS on these systems, but usually
they are starting from some prebuilt image. A major goal of this
project is to make it easy to build these images yourself. For this to
work, I’ve had to make lots of changes to NixOS cross-compilation
ecosystem, but the results seem to be very positive. I also want the
system to be locked down so that no user can login directly on the
machine. Instead, all administration is done on a remote machine, and
deployed through SSH and Nix remote builders.
&lt;/p&gt;

&lt;p&gt;
Right now, I have RetroArch (a frontend for a bunch of emulators) on
my Raspberry Pi 4, and Epiphany (a web browser) on my Raspberry Pi 0.
Both systems seem to be working pretty well.
&lt;/p&gt;

&lt;p&gt;
GitHub: &lt;a href=&quot;https://github.com/matthewbauer/nixiosk&quot;&gt;https://github.com/matthewbauer/nixiosk&lt;/a&gt;
&lt;/p&gt;

&lt;div class=&quot;outline-2&quot; id=&quot;outline-container-org11baea3&quot;&gt;
&lt;h2 id=&quot;org11baea3&quot;&gt;&lt;span class=&quot;section-number-2&quot;&gt;1&lt;/span&gt; Deploying&lt;/h2&gt;
&lt;div class=&quot;outline-text-2&quot; id=&quot;text-1&quot;&gt;
&lt;/div&gt;
&lt;div class=&quot;outline-3&quot; id=&quot;outline-container-org3936587&quot;&gt;
&lt;h3 id=&quot;org3936587&quot;&gt;&lt;span class=&quot;section-number-3&quot;&gt;1.1&lt;/span&gt; Install Nix&lt;/h3&gt;
&lt;div class=&quot;outline-text-3&quot; id=&quot;text-1-1&quot;&gt;
&lt;p&gt;
If you haven’t already, you need to install Nix. This can be done
through the installer:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-sh&quot;&gt;$ bash &amp;lt;(curl -L https://nixos.org/nix/install)
&lt;/pre&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;div class=&quot;outline-3&quot; id=&quot;outline-container-org9c45d30&quot;&gt;
&lt;h3 id=&quot;org9c45d30&quot;&gt;&lt;span class=&quot;section-number-3&quot;&gt;1.2&lt;/span&gt; Cache&lt;/h3&gt;
&lt;div class=&quot;outline-text-3&quot; id=&quot;text-1-2&quot;&gt;
&lt;p&gt;
To speed things up, you should setup a binary cache for nixiosk. This
can be done easily through &lt;a href=&quot;https://nixiosk.cachix.org/&quot;&gt;Cachix&lt;/a&gt;. First, install Cachix:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-sh&quot;&gt;$ nix-env -iA cachix -f https://cachix.org/api/v1/install
&lt;/pre&gt;
&lt;/div&gt;

&lt;p&gt;
Then, use the nixiosk cache:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-sh&quot;&gt;$ cachix use nixiosk
&lt;/pre&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;div class=&quot;outline-3&quot; id=&quot;outline-container-org16dc38e&quot;&gt;
&lt;h3 id=&quot;org16dc38e&quot;&gt;&lt;span class=&quot;section-number-3&quot;&gt;1.3&lt;/span&gt; Configuration&lt;/h3&gt;
&lt;div class=&quot;outline-text-3&quot; id=&quot;text-1-3&quot;&gt;
&lt;p&gt;
To make things simple, it just reads from an ad-hoc JSON file that
describe the hardware plus some other customizations. It looks like
this:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-json&quot;&gt;{
    &lt;span class=&quot;org-keyword&quot;&gt;&quot;hostName&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;nixiosk&quot;&lt;/span&gt;,
    &lt;span class=&quot;org-keyword&quot;&gt;&quot;hardware&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;raspberryPi4&quot;&lt;/span&gt;,
    &lt;span class=&quot;org-keyword&quot;&gt;&quot;authorizedKeys&quot;&lt;/span&gt;: [],
    &lt;span class=&quot;org-keyword&quot;&gt;&quot;program&quot;&lt;/span&gt;: {
        &lt;span class=&quot;org-keyword&quot;&gt;&quot;package&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;epiphany&quot;&lt;/span&gt;,
        &lt;span class=&quot;org-keyword&quot;&gt;&quot;executable&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;/bin/epiphany&quot;&lt;/span&gt;,
        &lt;span class=&quot;org-keyword&quot;&gt;&quot;args&quot;&lt;/span&gt;: [&lt;span class=&quot;org-string&quot;&gt;&quot;https://en.wikipedia.org/&quot;&lt;/span&gt;]
    },
    &lt;span class=&quot;org-keyword&quot;&gt;&quot;networks&quot;&lt;/span&gt;: {
        &lt;span class=&quot;org-keyword&quot;&gt;&quot;my-router&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;0000000000000000000000000000000000000000000000000000000000000000&quot;&lt;/span&gt;,
    },
    &lt;span class=&quot;org-keyword&quot;&gt;&quot;locale&quot;&lt;/span&gt;: {
        &lt;span class=&quot;org-keyword&quot;&gt;&quot;timeZone&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;America/New_York&quot;&lt;/span&gt;,
        &lt;span class=&quot;org-keyword&quot;&gt;&quot;regDom&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;US&quot;&lt;/span&gt;,
        &lt;span class=&quot;org-keyword&quot;&gt;&quot;lang&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;en_US.UTF-8&quot;&lt;/span&gt;
    },
    &lt;span class=&quot;org-keyword&quot;&gt;&quot;localSystem&quot;&lt;/span&gt;: {
        &lt;span class=&quot;org-keyword&quot;&gt;&quot;system&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;x86_64-linux&quot;&lt;/span&gt;,
        &lt;span class=&quot;org-keyword&quot;&gt;&quot;sshUser&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;me&quot;&lt;/span&gt;,
        &lt;span class=&quot;org-keyword&quot;&gt;&quot;hostName&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;my-laptop-host&quot;&lt;/span&gt;,
    }
}
&lt;/pre&gt;
&lt;/div&gt;

&lt;p&gt;
Here’s a basic idea of what each of these fields do:
&lt;/p&gt;

&lt;ul class=&quot;org-ul&quot;&gt;
&lt;li&gt;hostName: Name of the host to use. If mDNS is configured on your
network, this can be used to identify the IP address of the device
via “&amp;lt;hostName&amp;gt;.local”.&lt;/li&gt;
&lt;li&gt;hardware: A string describing what hardware we are using. Valid
values currently are “raspberryPi0”, “raspberryPi1”, “raspberryPi2”,
“raspberryPi3”, “raspberryPi4”.&lt;/li&gt;
&lt;li&gt;authorizedKeys: A list of SSH public keys that are authorized to
make changes to your device. Note this is required because no
passwords will be set for this system.&lt;/li&gt;
&lt;li&gt;program: What to do in the kiosk. This should be a Nixpkgs attribute
(&lt;b&gt;package&lt;/b&gt;), an &lt;b&gt;executable&lt;/b&gt; in that package, and a list of &lt;b&gt;args&lt;/b&gt;.&lt;/li&gt;
&lt;li&gt;networks: This is a name/value pairing of SSIDs to PSK passphrases.
This can be found with the wpa_passphrase(8) command from
wpa_supplicant.&lt;/li&gt;
&lt;li&gt;locale: This provides some information of what localizations to use.
You can set &lt;a href=&quot;https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2&quot;&gt;regulation domain&lt;/a&gt;, &lt;a href=&quot;https://www.gnu.org/software/libc/manual/html_node/Locale-Names.html#Locale-Names&quot;&gt;language&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_tz_database_time_zones&quot;&gt;time zone&lt;/a&gt; via “regDom”,
“lang”, and “timeZone”. If unspecified, defaults to US / English /
New York.&lt;/li&gt;
&lt;li&gt;localSystem: Information on system to use for &lt;a href=&quot;https://github.com/matthewbauer/nixiosk#remote-builder-optional&quot;&gt;remote builder&lt;/a&gt;.
Optional.&lt;/li&gt;
&lt;/ul&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;div class=&quot;outline-3&quot; id=&quot;outline-container-orgddeb048&quot;&gt;
&lt;h3 id=&quot;orgddeb048&quot;&gt;&lt;span class=&quot;section-number-3&quot;&gt;1.4&lt;/span&gt; Initial deployment&lt;/h3&gt;
&lt;div class=&quot;outline-text-3&quot; id=&quot;text-1-4&quot;&gt;
&lt;p&gt;
The deployment is pretty easy provided you have &lt;a href=&quot;https://nixos.org/nix/&quot;&gt;Nix installed&lt;/a&gt;. Here
are some steps:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-sh&quot;&gt;$ git clone https://github.com/matthewbauer/nixiosk.git
$ cd nixiosk/
$ cp nixiosk.json.sample nixiosk.json
&lt;/pre&gt;
&lt;/div&gt;

&lt;p&gt;
Now you need to make some changes to nixiosk.json to reflect what you
want your system to do. The important ones are ‘authorizedKeys’ and
‘networks’ so that your systems can startup and you can connect to it.
&lt;/p&gt;

&lt;p&gt;
If you have an SSH key setup, you can get its value with:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-sh&quot;&gt;$ cat $&lt;span class=&quot;org-variable-name&quot;&gt;HOME&lt;/span&gt;/.ssh/id_rsa.pub
&lt;span class=&quot;org-whitespace-line&quot;&gt;ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC050iPG8ckY/dj2O3ol20G2lTdr7ERFz4LD3R4yqoT5W0THjNFdCqavvduCIAtF1Xx/OmTISblnGKf10rYLNzDdyMMFy7tUSiC7/T37EW0s+EFGhS9yOcjCVvHYwgnGZCF4ec33toE8Htq2UKBVgtE0PMwPAyCGYhFxFLYN8J8/xnMNGqNE6iTGbK5qb4yg3rwyrKMXLNGVNsPVcMfdyk3xqUilDp4U7HHQpqX0wKrUvrBZ87LnO9z3X/QIRVQhS5GqnIjRYe4L9yxZtTjW5HdwIq1jcvZc/1Uu7bkMh3gkCwbrpmudSGpdUlyEreaHOJf3XH4psr6IMGVJvxnGiV9 mbauer@dellbook&lt;/span&gt;
&lt;/pre&gt;
&lt;/div&gt;

&lt;p&gt;
which will give you a line for “authorizedKeys” like:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-json&quot;&gt;&lt;span class=&quot;org-keyword&quot;&gt;&lt;span class=&quot;org-whitespace-line&quot;&gt;&quot;authorizedKeys&quot;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;org-whitespace-line&quot;&gt;: [&lt;/span&gt;&lt;span class=&quot;org-string&quot;&gt;&lt;span class=&quot;org-whitespace-line&quot;&gt;&quot;ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC050iPG8ckY/dj2O3ol20G2lTdr7ERFz4LD3R4yqoT5W0THjNFdCqavvduCIAtF1Xx/OmTISblnGKf10rYLNzDdyMMFy7tUSiC7/T37EW0s+EFGhS9yOcjCVvHYwgnGZCF4ec33toE8Htq2UKBVgtE0PMwPAyCGYhFxFLYN8J8/xnMNGqNE6iTGbK5qb4yg3rwyrKMXLNGVNsPVcMfdyk3xqUilDp4U7HHQpqX0wKrUvrBZ87LnO9z3X/QIRVQhS5GqnIjRYe4L9yxZtTjW5HdwIq1jcvZc/1Uu7bkMh3gkCwbrpmudSGpdUlyEreaHOJf3XH4psr6IMGVJvxnGiV9 mbauer@dellbook&quot;&lt;/span&gt;&lt;/span&gt;&lt;span class=&quot;org-whitespace-line&quot;&gt;],&lt;/span&gt;
&lt;/pre&gt;
&lt;/div&gt;

&lt;p&gt;
and you can get a PSK value for your WiFi network with:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-sh&quot;&gt;$ nix run nixpkgs.wpa_supplicant -c wpa_passphrase my-network
&lt;span class=&quot;org-variable-name&quot;&gt;network&lt;/span&gt;={
        &lt;span class=&quot;org-variable-name&quot;&gt;ssid&lt;/span&gt;=&lt;span class=&quot;org-string&quot;&gt;&quot;my-network&quot;&lt;/span&gt;
        &lt;span class=&quot;org-comment-delimiter&quot;&gt;#&lt;/span&gt;&lt;span class=&quot;org-comment&quot;&gt;psk=&quot;abcdefgh&quot;&lt;/span&gt;
        &lt;span class=&quot;org-variable-name&quot;&gt;psk&lt;/span&gt;=17e76a6490ac112dbeba996caa7cd1387c6ebf6ce721ef704f92b681bb2e9000
}
&lt;/pre&gt;
&lt;/div&gt;

&lt;p&gt;
so your .json file looks like:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-json&quot;&gt;&lt;span class=&quot;org-keyword&quot;&gt;&quot;networks&quot;&lt;/span&gt;: {
  &lt;span class=&quot;org-keyword&quot;&gt;&quot;my-network&quot;&lt;/span&gt;: &lt;span class=&quot;org-string&quot;&gt;&quot;17e76a6490ac112dbeba996caa7cd1387c6ebf6ce721ef704f92b681bb2e9000&quot;&lt;/span&gt;,
},
&lt;/pre&gt;
&lt;/div&gt;

&lt;p&gt;
Now, after inserting your Raspberry Pi SD card into the primary slot,
you can deploy to it with:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-sh&quot;&gt;$ ./deploy.sh /dev/mmcblk0
&lt;/pre&gt;
&lt;/div&gt;

&lt;p&gt;
You can now eject your SD card and insert it into your Raspberry Pi.
It will boot immediately to an Epiphany browser, loading
en.wikipedia.org.
&lt;/p&gt;

&lt;p&gt;
&lt;a href=&quot;https://github.com/matthewbauer/nixiosk#troubleshooting&quot;&gt;Troubleshooting steps&lt;/a&gt; can be found in the README.
&lt;/p&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;div class=&quot;outline-3&quot; id=&quot;outline-container-orgefabaf8&quot;&gt;
&lt;h3 id=&quot;orgefabaf8&quot;&gt;&lt;span class=&quot;section-number-3&quot;&gt;1.5&lt;/span&gt; Redeployments&lt;/h3&gt;
&lt;div class=&quot;outline-text-3&quot; id=&quot;text-1-5&quot;&gt;
&lt;p&gt;
You can pretty easily make changes to a running system given you have
SSH access. This is as easy as cloning the running config:
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-sh&quot;&gt;$ git clone ssh://root@nixiosk.local/etc/nixos/configuration.git nixiosk-configuration
$ cd nixiosk-configuration
&lt;/pre&gt;
&lt;/div&gt;

&lt;p&gt;
Then, make some changes in your repo. After your done, you can just
run ‘git push’ to redeploy.
&lt;/p&gt;

&lt;div class=&quot;org-src-container&quot;&gt;
&lt;pre class=&quot;src src-sh&quot;&gt;$ git add .
$ git commit
$ git push
&lt;/pre&gt;
&lt;/div&gt;

&lt;p&gt;
You’ll see the NixOS switch-to-configuration log in your command
output. If all is successful, the system should immediately reflect
your changes. If not, the output of Git should explain what went
wrong.
&lt;/p&gt;

&lt;p&gt;
Note, that some versions of the Raspberry Pi like the 0 and the 1 are
not big enough to redeploy the whole system. You will probably need to
setup remote builders. This is &lt;a href=&quot;https://github.com/matthewbauer/nixiosk#remote-builder-optional&quot;&gt;described in the README&lt;/a&gt;.
&lt;/p&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;div class=&quot;outline-2&quot; id=&quot;outline-container-org6df267f&quot;&gt;
&lt;h2 id=&quot;org6df267f&quot;&gt;&lt;span class=&quot;section-number-2&quot;&gt;2&lt;/span&gt; Technology&lt;/h2&gt;
&lt;div class=&quot;outline-text-2&quot; id=&quot;text-2&quot;&gt;
&lt;p&gt;
Here are some of the pieces that make the Kiosk system possible:
&lt;/p&gt;

&lt;ul class=&quot;org-ul&quot;&gt;
&lt;li&gt;&lt;a href=&quot;https://www.hjdskes.nl/projects/cage/&quot;&gt;Cage&lt;/a&gt; / &lt;a href=&quot;https://wayland.freedesktop.org/&quot;&gt;Wayland&lt;/a&gt;: Cage is a Wayland compositor that allows only one
application to display at a time. This makes the system a true
Kiosk.&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://nixos.org/&quot;&gt;NixOS&lt;/a&gt; - A Linux distro built on top of functional package management.&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://gitlab.com/obsidian.systems/basalt/&quot;&gt;Basalt&lt;/a&gt;: A tool to manage NixOS directly from Git. This allows doing
push-to-deploy directly to NixOS.&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.freedesktop.org/wiki/Software/Plymouth/&quot;&gt;Plymouth&lt;/a&gt;: Nice graphical boot animations. Right now, it uses the
NixOS logo but in the future this should be configurable so that you
can include your own branding.&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.openssh.com/&quot;&gt;OpenSSH&lt;/a&gt;: Since no direct login is available, SSH is required for
remote administration.&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;http://www.avahi.org/&quot;&gt;Avahi&lt;/a&gt;: Configures mDNS registration for the system, allowing you to
remember host names instead of IP addresses.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;
I would also like to include some more tools to make administration
easier:
&lt;/p&gt;

&lt;ul class=&quot;org-ul&quot;&gt;
&lt;li&gt;ddclient / miniupnp: Allow registering external IP address with a
DNS provider. This would enable administration outside of the
device’s immediate network.&lt;/li&gt;
&lt;/ul&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;div class=&quot;outline-2&quot; id=&quot;outline-container-org6777f70&quot;&gt;
&lt;h2 id=&quot;org6777f70&quot;&gt;&lt;span class=&quot;section-number-2&quot;&gt;3&lt;/span&gt; Project&lt;/h2&gt;
&lt;div class=&quot;outline-text-2&quot; id=&quot;text-3&quot;&gt;
&lt;p&gt;
You can try it out right now if you have an Raspberry Pi system. Other
hardware is probably not too hard, but may require tweaking. The
project page is available at &lt;a href=&quot;https://github.com/matthewbauer/nixiosk&quot;&gt;https://github.com/matthewbauer/nixiosk&lt;/a&gt;
and issues and pull requests are welcomed.
&lt;/p&gt;
&lt;/div&gt;
&lt;/div&gt;</description>
	<pubDate>Mon, 23 Mar 2020 00:00:00 +0000</pubDate>
</item>
<item>
	<title>Cachix: Proposal for improving Nix error messages</title>
	<guid isPermaLink="true">https://blog.cachix.org/post/2020-03-18-proposal-for-improving-nix-error-messages/</guid>
	<link>https://blog.cachix.org/post/2020-03-18-proposal-for-improving-nix-error-messages/</link>
	<description>I’m lucky to be in touch with a lot of people that use Nix day to day.
One of the most occouring annoyances that pops up more frequently with those starting with Nix are confusing error messages.
Since Nix community has previously succesfully stepped up and funded removal of Perl to reduce barriers for source code contributions, I think we ought to do the same for removing barriers when using Nix.</description>
	<pubDate>Wed, 18 Mar 2020 08:00:00 +0000</pubDate>
	<author>support@cachix.org (Domen Kožar)</author>
</item>
<item>
	<title>Flying Circus: Our new NixOS 19.03 Platform Is Ready for Production</title>
	<guid isPermaLink="false">http://blog.flyingcircus.io/?p=5295</guid>
	<link>https://blog.flyingcircus.io/2020/02/28/our-new-nixos-19-03-platform-is-ready-for-production/</link>
	<description>&lt;p&gt;We have developed our third-generation platform which is now based on NixOS 19.03. All provided components have been ported to the new platform and VMs are already running in production.&lt;/p&gt;



&lt;p&gt;Most of our development work is done for the new platform and new features will be available only for it. We pull in security updates from upstream regularly and will follow new NixOS releases more quickly in the future. The old NixOS 15.09 platform still receives critical security and bug fixes.&lt;/p&gt;



&lt;p&gt;Effective March 6, VMs created via customer self-service will use the 19.03 platform.&lt;/p&gt;



&lt;p&gt;You can find the documentation for the new platform here:&lt;/p&gt;



&lt;p&gt;&lt;a href=&quot;https://flyingcircus.io/doc/guide/platform_nixos_2/index.html&quot;&gt;https://flyingcircus.io/doc/guide/platform_nixos_2/index.html&lt;/a&gt;&lt;/p&gt;



&lt;p&gt;We recommend user profiles (done with buildEnv) in case your application needs specific packages in its environment:&lt;/p&gt;



&lt;p&gt;&lt;a href=&quot;https://flyingcircus.io/doc/guide/platform_nixos_2/user_profile.html&quot;&gt;https://flyingcircus.io/doc/guide/platform_nixos_2/user_profile.html&lt;/a&gt;&lt;/p&gt;



&lt;h2&gt;Upgrading 15.09 Machines&lt;/h2&gt;



&lt;p&gt;Upgrading existing VMs online is supported and we have already done that for a number of VMs.&lt;br /&gt;Sometimes however, it can be better to create new NixOS VMs in parallel and set up your applications there.&lt;/p&gt;



&lt;p&gt;Most managed components will just work after the upgrade. We are working on instructions for specific things that should be done before or after the upgrade.&lt;/p&gt;



&lt;p&gt;If you’re a customer with a support contract in the “Guided” or “Managed” service classes&lt;br /&gt;then we’ll approach you directly and discuss when and how to upgrade VMs in the coming months.&lt;/p&gt;



&lt;p&gt;If you’re a customer in the “Hosted” service class then we recommend contacting our support team to discuss the upgrade.&lt;/p&gt;



&lt;h2&gt;If you have questions …&lt;/h2&gt;



&lt;p&gt;As always: if you have any questions or comments then let us know and send us an email to &lt;a href=&quot;mailto:support@flyingcircus.io&quot;&gt;support@flyingcircus.io&lt;/a&gt;.&lt;/p&gt;</description>
	<pubDate>Fri, 28 Feb 2020 14:16:49 +0000</pubDate>
</item>
<item>
	<title>nixbuild.net: Introducing nixbuild.net</title>
	<guid isPermaLink="true">https://blog.nixbuild.net/posts/2020-02-18-introducing-nixbuild-net.html</guid>
	<link>https://blog.nixbuild.net/posts/2020-02-18-introducing-nixbuild-net.html</link>
	<description>&lt;p&gt;Exactly one month ago, I &lt;a href=&quot;https://discourse.nixos.org/t/announcing-nixbuild-net-nix-build-as-a-service&quot;&gt;announced&lt;/a&gt; the &lt;a href=&quot;https://nixbuild.net&quot;&gt;nixbuild.net&lt;/a&gt; service. Since then, there have been lots of work on functionality, performance and stability of the service. As of today, nixbuild.net is exiting alpha and entering private beta phase. If you want to try it out, just &lt;a href=&quot;mailto:rickard@nixbuild.net&quot;&gt;send me an email&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Today, I’m also launching the &lt;a href=&quot;https://blog.nixbuild.net&quot;&gt;nixbuild.net blog&lt;/a&gt;, which is intended as an outlet for anything related to the nixbuild.net service. Announcements, demos, technical articles and various tips and tricks. We’ll start out with a proper introduction of nixbuild.net; why it was built, what it can help you with and what the long-term goals are.&lt;/p&gt;

&lt;h2 id=&quot;why-nixbuild.net&quot;&gt;Why nixbuild.net?&lt;/h2&gt;
&lt;p&gt;&lt;a href=&quot;https://nixos.org/nix/&quot;&gt;Nix&lt;/a&gt; has great built-in support for &lt;a href=&quot;https://nixos.org/nix/manual/#chap-distributed-builds&quot;&gt;distributing builds&lt;/a&gt; to remote machines. You just need to setup a standard Nix enviroment on your build machines, and make sure they are accessible via SSH. Just like that, you can offload your heavy builds to a couple of beefy build servers, saving your poor laptop’s fan from spinning up.&lt;/p&gt;
&lt;p&gt;However, just when you’ve tasted those sweet distributed builds you very likely run into the issue of &lt;em&gt;scaling&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;What if you need a really big server to run your builds, but only really need it once or twice per day? You’ll be wasting a lot of money keeping that build server available.&lt;/p&gt;
&lt;p&gt;And what if you occasionally have lots and lots of builds to run, or if your whole development team wants to share the build servers? Then you probably need to add more build servers, which means more wasted money when they are not used.&lt;/p&gt;
&lt;p&gt;So, you start looking into auto-scaling your build servers. This is quite easy to do if you use some cloud provider like AWS, Azure or GCP. But, this is where Nix will stop cooperating with you. It is really tricky to get Nix to work nicely together with an auto-scaled set of remote build machines. Nix has only a very coarse view of the “current load” of a build machine and can therefore not make very informed decisions on exactly how to distribute the builds. If there are multiple Nix instances (one for each developer in your team) fighting for the same resources, things get even trickier. It is really easy to end up in a situation where a bunch of really heavy builds are fighting for CPU time on the same build server while the other servers are idle or running lightweight build jobs.&lt;/p&gt;
&lt;p&gt;If you use &lt;a href=&quot;https://nixos.org/hydra/&quot;&gt;Hydra&lt;/a&gt;, the continous build system for Nix, you can find scripts for using auto-scaled AWS instances, but it is still tricky to set it up. And in the end, it doesn’t work perfectly since Nix/Hydra has no notion of “consumable” CPU/memory resources so the build scheduling is somewhat hit-and-miss.&lt;/p&gt;
&lt;p&gt;Even if you manage to come up with a solution that can handle your workload in an acceptable manner, you now have a new job: &lt;em&gt;maintaining&lt;/em&gt; uniquely configured build servers. Possibly for your whole company.&lt;/p&gt;
&lt;p&gt;Through my consulting company, &lt;a href=&quot;https://immutablesolutions.com/&quot;&gt;Immutable Solutions&lt;/a&gt;, I’ve done a lot of work on Nix-based deployments, and I’ve always struggled with half-baked solutions to the Nix build farm problem. This is how the idea of the nixbuild.net service was born — a service that can fill in the missing pieces of the Nix distributed build puzzle and package it as a simple, no-maintenance, cost-effective service.&lt;/p&gt;
&lt;h2 id=&quot;who-are-we&quot;&gt;Who are We?&lt;/h2&gt;
&lt;p&gt;nixbuild.net is developed and operated by me (Rickard Nilsson) and my colleague David Waern. We both have extensive experience in building Nix-based solutions, for ourselves and for various clients.&lt;/p&gt;
&lt;p&gt;We’re bootstrapping nixbuild.net, and we are long-term committed to keep developing and operating the service. Today, nixbuild.net can be productively used for its main purpose — running Nix builds in a scalable and cost-effective way — but there are lots of things that can (and will) be built on top of and around that core. Read more about this below.&lt;/p&gt;
&lt;h2 id=&quot;what-does-nixbuild.net-look-like&quot;&gt;What does nixbuild.net Look Like?&lt;/h2&gt;
&lt;p&gt;To the end-user, a person or team using Nix for building software, nixbuild.net behaves just like any other &lt;a href=&quot;https://nixos.org/nix/manual/#chap-distributed-builds&quot;&gt;remote build machine&lt;/a&gt;. As such, you can add it as an entry in your &lt;code&gt;/etc/nix/machines&lt;/code&gt; file:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;beta.nixbuild.net x86_64-linux - 100 1 big-parallel,benchmark&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;big-parallel,benchmark&lt;/code&gt; assignment is something that is called &lt;em&gt;system features&lt;/em&gt; in Nix. You can use that as a primitive scheduling strategy if you have multiple remote machines. Nix will only submit builds that have been marked as requiring a specific system feature to machines that are assigned that feature.&lt;/p&gt;
&lt;p&gt;The number 100 in the file above tells Nix that it is allowed to submit up to 100 simultaneous builds to &lt;code&gt;beta.nixbuild.net&lt;/code&gt;. Usually, you use this property to balance builds between remote machines, and to make sure that a machine doesn’t run too many builds at the same time. This works OK when you have rather homogeneous builds, and only one single Nix client is using a set of build servers. If multiple Nix clients use the same set of build servers, this simplistic scheduling breaks down, since a given Nix client loses track on how many builds are really running on a server.&lt;/p&gt;
&lt;p&gt;However, when you’re using nixbuild.net, you can set this number to anything really, since nixbuild.net will take care of the scheduling and scaling on its own, and it will not let multiple Nix clients step on each other’s toes. In fact each build that nixbuild.net runs is securely isolated from other builds and by default gets exclusive access to the resources (CPU and memory) it has been assigned.&lt;/p&gt;
&lt;p&gt;Apart from setting up the distributed Nix machines, you need to configure SSH. When you register an account on nixbuild.net, you’ll provide us with a public SSH key. The corresponding private key is used for connecting to nixbuild.net. This private key needs to be readable by the user that runs the Nix build. This is usually the &lt;code&gt;root&lt;/code&gt; user, if you have a standard Nix setup where the &lt;code&gt;nix-daemon&lt;/code&gt; process runs as the root user.&lt;/p&gt;
&lt;p&gt;That’s all there is to it, now we can run builds using nixbuild.net!&lt;/p&gt;
&lt;p&gt;Let’s try building the following silly build, just so we can see some action:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;let pkgs = import &amp;lt;nixpkgs&amp;gt; { system = &quot;x86_64-linux&quot;; };

in pkgs.runCommand &quot;silly&quot; {} ''
  n=0
  while (($n &amp;lt; 12)); do
    date | tee -a $out
    sleep 10
    n=$(($n + 1))
  done
''&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This build will run for 2 minutes and output the current date every ten seconds:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;$ nix-build silly.nix
these derivations will be built:
  /nix/store/cy14fc13d3nzl65qp0sywvbjnnl48jf8-silly.drv
building '/nix/store/khvphdj3q7nyim46jk97fjp174damrik-silly.drv' on 'ssh://beta.nixbuild.net'...
Mon Feb 17 20:53:47 UTC 2020
Mon Feb 17 20:53:57 UTC 2020
Mon Feb 17 20:54:07 UTC 2020&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can see that Nix is telling us that the build is running on nixbuild.net!&lt;/p&gt;
&lt;h3 id=&quot;the-nixbuild.net-shell&quot;&gt;The nixbuild.net Shell&lt;/h3&gt;
&lt;p&gt;nixbuild.net supports a simple shell interface that you can access through SSH. This shell allows you to retrieve information about your builds on the service.&lt;/p&gt;
&lt;p&gt;For example, we can list the currently running builds:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;$ ssh beta.nixbuild.net shell
nixbuild.net&amp;gt; list builds --running
10524 2020-02-17 21:05:20Z [40.95s] [Running]
      /nix/store/khvphdj3q7nyim46jk97fjp174damrik-silly.drv&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We can also get information about any derivation or nix store path that has been built:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;nixbuild.net&amp;gt; show drv /nix/store/khvphdj3q7nyim46jk97fjp174damrik-silly.drv
Derivation
  path = /nix/store/khvphdj3q7nyim46jk97fjp174damrik-silly.drv
  builds = 1
  successful builds = 1

Outputs
  out -&amp;gt; /nix/store/8c7sndr3npwmskj9zzp4347cnqh5p8q0-silly

Builds
  10524 2020-02-17 21:05:20Z [02:01] [Built]&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This shell is under development, and new features are added continuously. A web-based frontend will also be implemented.&lt;/p&gt;
&lt;h2 id=&quot;the-road-ahead&quot;&gt;The Road Ahead&lt;/h2&gt;
&lt;p&gt;To finish up this short introduction to nixbuild.net, let’s talk a bit about our long-term goals for the service.&lt;/p&gt;
&lt;p&gt;The core purpose of nixbuild.net is to provide Nix users with pay-per-use distributed builds that are simple to set up and integrate into any workflow. The build execution should be performant and secure.&lt;/p&gt;
&lt;p&gt;There are a number of features that basically just are nice side-effects of the design of nixbuild.net:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Building a large number of variants of the same derivation (a build matrix or some sort of parameter sweep) will take the same time as running a single build, since nixbuild.net can run all builds in parallel.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Running repeated builds to find issues related to non-determinism/reproducability will not take longer than running a single build.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;A whole team/company can share the same account in nixbuild.net letting builds be shared in a cost-effective way. If everyone in a team delegates builds to nixbuild.net, the same derivation will never have to be built twice. This is similar to having a shared Nix cache, but avoids having to configure a cache and perform network uploads for each build artifact. Of course, nixbuild.net can be combined with a Nix cache too, if desired.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Beyond the above we have lots of thoughts on where we want to take nixbuild.net. I’m not going to enumerate possible directions here and now, but one big area that nixbuild.net is particularly suited for is advanced build analysis and visualisation. The sandbox that has been developed to securely isolate builds from each other also gives us a unique way to analyze exactly how a build behaves. One can imagine nixbuild.net being able give very detailed feedback to users about build bottlenecks, performance regressions, unused dependencies etc.&lt;/p&gt;
&lt;p&gt;With that said, our primary focus right now is to make nixbuild.net a robust workhorse for your Nix builds, enabling you to fully embrace Nix without being limited by local compute resources. Please &lt;a href=&quot;mailto:rickard@nixbuild.net&quot;&gt;get in touch&lt;/a&gt; if you want try out nixbuild.net, or if you have any questions or comments!&lt;/p&gt;</description>
	<pubDate>Tue, 18 Feb 2020 00:00:00 +0000</pubDate>
	<author>support@nixbuild.net (nixbuild.net)</author>
</item>
<item>
	<title>Sander van der Burg: A declarative process manager-agnostic deployment framework based on Nix tooling</title>
	<guid isPermaLink="false">tag:blogger.com,1999:blog-1397115249631682228.post-3829850759126756827</guid>
	<link>http://sandervanderburg.blogspot.com/2020/02/a-declarative-process-manager-agnostic.html</link>
	<description>In a previous blog post written two months ago, &lt;a href=&quot;https://sandervanderburg.blogspot.com/2019/11/a-nix-based-functional-organization-for.html&quot;&gt;I have introduced a new experimental Nix-based process framework&lt;/a&gt;, that provides the following features:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;It uses the &lt;strong&gt;Nix expression language&lt;/strong&gt; for configuring running process instances, including their dependencies. The configuration process is based on only a few &lt;strong&gt;simple concepts&lt;/strong&gt;: function definitions to define constructors that generate process manager configurations, function invocations to compose running process instances, and &lt;a href=&quot;https://sandervanderburg.blogspot.com/2013/09/managing-user-environments-with-nix.html&quot;&gt;Nix profiles&lt;/a&gt; to make collections of process configurations accessible from a single location.&lt;/li&gt;&lt;li&gt;The &lt;strong&gt;Nix package manager&lt;/strong&gt; delivers all packages and configuration files and isolates them in the Nix store, so that they never conflict with other running processes and packages.&lt;/li&gt;&lt;li&gt;It identifies &lt;strong&gt;process dependencies&lt;/strong&gt;, so that a process manager can ensure that processes are activated and deactivated in the right order.&lt;/li&gt;&lt;li&gt;The ability to deploy &lt;strong&gt;multiple instances&lt;/strong&gt; of the same process, by making conflicting resources configurable.&lt;/li&gt;&lt;li&gt;Deploying processes/services as an &lt;strong&gt;unprivileged user&lt;/strong&gt;.&lt;/li&gt;&lt;li&gt;Advanced concepts and features, such as &lt;a href=&quot;http://man7.org/linux/man-pages/man7/namespaces.7.html&quot;&gt;namespaces&lt;/a&gt; and &lt;a href=&quot;http://man7.org/linux/man-pages/man7/cgroups.7.html&quot;&gt;cgroups&lt;/a&gt;, are &lt;strong&gt;not required&lt;/strong&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;Another objective of the framework is that it should work with a variety of process managers on a variety of operating systems.&lt;br /&gt;&lt;br /&gt;In my previous blog post, I was deliberately using sysvinit scripts (also known as LSB Init compliant scripts) to manage the lifecycle of running processes as a starting point, because they are universally supported on Linux and self contained -- sysvinit scripts only require the right packages installed, but they do not rely on external programs that manage the processes' life-cycle. Moreover, sysvinit scripts can also be conveniently used as an unprivileged user.&lt;br /&gt;&lt;br /&gt;I have also developed a Nix function that can be used to more conveniently generate sysvinit scripts. Traditionally, these scripts are written by hand and basically require that the implementer writes the same boilerplate code over and over again, such as the activities that start and stop the process.&lt;br /&gt;&lt;br /&gt;The sysvinit script generator function can also be used to directly specify the implementation of all activities that manage the life-cycle of a process, such as:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{createSystemVInitScript, nginx, stateDir}:&lt;br /&gt;{configFile, dependencies ? [], instanceSuffix ? &quot;&quot;}:&lt;br /&gt;&lt;br /&gt;let&lt;br /&gt;  instanceName = &quot;nginx${instanceSuffix}&quot;;&lt;br /&gt;  nginxLogDir = &quot;${stateDir}/${instanceName}/logs&quot;;&lt;br /&gt;in&lt;br /&gt;createSystemVInitScript {&lt;br /&gt;  name = instanceName;&lt;br /&gt;  description = &quot;Nginx&quot;;&lt;br /&gt;  activities = {&lt;br /&gt;    start = ''&lt;br /&gt;      mkdir -p ${nginxLogDir}&lt;br /&gt;      log_info_msg &quot;Starting Nginx...&quot;&lt;br /&gt;      loadproc ${nginx}/bin/nginx -c ${configFile} -p ${stateDir}&lt;br /&gt;      evaluate_retval&lt;br /&gt;    '';&lt;br /&gt;    stop = ''&lt;br /&gt;      log_info_msg &quot;Stopping Nginx...&quot;&lt;br /&gt;      killproc ${nginx}/bin/nginx&lt;br /&gt;      evaluate_retval&lt;br /&gt;    '';&lt;br /&gt;    reload = ''&lt;br /&gt;      log_info_msg &quot;Reloading Nginx...&quot;&lt;br /&gt;      killproc ${nginx}/bin/nginx -HUP&lt;br /&gt;      evaluate_retval&lt;br /&gt;    '';&lt;br /&gt;    restart = ''&lt;br /&gt;      $0 stop&lt;br /&gt;      sleep 1&lt;br /&gt;      $0 start&lt;br /&gt;    '';&lt;br /&gt;    status = &quot;statusproc ${nginx}/bin/nginx&quot;;&lt;br /&gt;  };&lt;br /&gt;  runlevels = [ 3 4 5 ];&lt;br /&gt;&lt;br /&gt;  inherit dependencies instanceName;&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;In the above Nix expression, we specify five activities to manage the life-cycle of Nginx, a free/open source web server:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;The &lt;strong&gt;start&lt;/strong&gt; activity initializes the state of Nginx and starts the process (&lt;a href=&quot;https://sandervanderburg.blogspot.com/2020/01/writing-well-behaving-daemon-in-c.html&quot;&gt;as a daemon&lt;/a&gt; that runs in the background).&lt;/li&gt;&lt;li&gt;&lt;strong&gt;stop&lt;/strong&gt; stops the Nginx daemon.&lt;/li&gt;&lt;li&gt;&lt;strong&gt;reload&lt;/strong&gt; instructs Nginx to reload its configuration&lt;/li&gt;&lt;li&gt;&lt;strong&gt;restart&lt;/strong&gt; restarts the process&lt;/li&gt;&lt;li&gt;&lt;strong&gt;status&lt;/strong&gt; shows whether the process is running or not.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;Besides directly implementing activities, the Nix function invocation shown above can also be used on a much &lt;strong&gt;higher level&lt;/strong&gt; -- typically, sysvinit scripts follow the same conventions. Nearly all sysvinit scripts implement the activities described above to manage the life-cycle of a process, and these typically need to be re-implemented over and over again.&lt;br /&gt;&lt;br /&gt;We can also generate the implementations of these activities automatically from a high level specification, such as:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{createSystemVInitScript, nginx,  stateDir}:&lt;br /&gt;{configFile, dependencies ? [], instanceSuffix ? &quot;&quot;}:&lt;br /&gt;&lt;br /&gt;let&lt;br /&gt;  instanceName = &quot;nginx${instanceSuffix}&quot;;&lt;br /&gt;  nginxLogDir = &quot;${stateDir}/${instanceName}/logs&quot;;&lt;br /&gt;in&lt;br /&gt;createSystemVInitScript {&lt;br /&gt;  name = instanceName;&lt;br /&gt;  description = &quot;Nginx&quot;;&lt;br /&gt;  initialize = ''&lt;br /&gt;    mkdir -p ${nginxLogDir}&lt;br /&gt;  '';&lt;br /&gt;  process = &quot;${nginx}/bin/nginx&quot;;&lt;br /&gt;  args = [ &quot;-c&quot; configFile &quot;-p&quot; stateDir ];&lt;br /&gt;  runlevels = [ 3 4 5 ];&lt;br /&gt;&lt;br /&gt;  inherit dependencies instanceName;&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;You could basically say that the above &lt;i&gt;createSystemVInitScript&lt;/i&gt; function invocation makes the configuration process of a sysvinit script &quot;&lt;a href=&quot;https://sandervanderburg.blogspot.com/2016/03/the-nixos-project-and-deploying-systems.html&quot;&gt;&lt;strong&gt;more declarative&lt;/strong&gt;&lt;/a&gt;&quot; -- you do not need to specify the activities that need to be executed to manage processes, but instead, you specify the &lt;strong&gt;relevant characteristics&lt;/strong&gt; of a running process.&lt;br /&gt;&lt;br /&gt;From this high level specification, the implementations for all required activities will be derived, using conventions that are commonly used to write sysvinit scripts.&lt;br /&gt;&lt;br /&gt;After completing the initial version of the process management framework that works with sysvinit scripts, I have also been investigating other process managers. I discovered that their configuration processes have many things in common with the sysvinit approach. As a result, I have decided to explore these declarative deployment concepts a bit further.&lt;br /&gt;&lt;br /&gt;In this blog post, I will describe a declarative process manager-agnostic deployment approach that we can integrate into the experimental Nix-based process management framework.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;Writing declarative deployment specifications for managed running processes&lt;/h2&gt;&lt;br /&gt;As explained in the introduction, I have also been experimenting with other process managers than sysvinit. For example, instead of generating a sysvinit script that manages the life-cycle of a process, such as the Nginx server, we can also generate a supervisord configuration file to define Nginx as a program that can be managed with supervisord:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{createSupervisordProgram, nginx, stateDir}:&lt;br /&gt;{configFile, dependencies ? [], instanceSuffix ? &quot;&quot;}:&lt;br /&gt;&lt;br /&gt;let&lt;br /&gt;  instanceName = &quot;nginx${instanceSuffix}&quot;;&lt;br /&gt;  nginxLogDir = &quot;${stateDir}/${instanceName}/logs&quot;;&lt;br /&gt;in&lt;br /&gt;createSupervisordProgram {&lt;br /&gt;  name = instanceName;&lt;br /&gt;  command = &quot;mkdir -p ${nginxLogDir}; &quot;+&lt;br /&gt;    &quot;${nginx}/bin/nginx -c ${configFile} -p ${stateDir}&quot;;&lt;br /&gt;  inherit dependencies;&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Invoking the above function will generate a supervisord program configuration file, instead of a sysvinit script.&lt;br /&gt;&lt;br /&gt;With the following Nix expression, we can generate a systemd unit file so that Nginx's life-cycle can be managed by systemd:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{createSystemdService, nginx, stateDir}:&lt;br /&gt;{configFile, dependencies ? [], instanceSuffix ? &quot;&quot;}:&lt;br /&gt;&lt;br /&gt;let&lt;br /&gt;  instanceName = &quot;nginx${instanceSuffix}&quot;;&lt;br /&gt;  nginxLogDir = &quot;${stateDir}/${instanceName}/logs&quot;;&lt;br /&gt;in&lt;br /&gt;createSystemdService {&lt;br /&gt;  name = instanceName;&lt;br /&gt;  Unit = {&lt;br /&gt;    Description = &quot;Nginx&quot;;&lt;br /&gt;  };&lt;br /&gt;  Service = {&lt;br /&gt;    ExecStartPre = &quot;+mkdir -p ${nginxLogDir}&quot;;&lt;br /&gt;    ExecStart = &quot;${nginx}/bin/nginx -c ${configFile} -p ${stateDir}&quot;;&lt;br /&gt;    Type = &quot;simple&quot;;&lt;br /&gt;  };&lt;br /&gt;&lt;br /&gt;  inherit dependencies;&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;What you may probably notice when comparing the above two Nix expressions with the last sysvinit example (that captures process characteristics instead of activities), is that they all contain very similar properties. Their main difference is a slightly different organization and naming convention, because each abstraction function is tailored towards the configuration conventions that each target process manager uses.&lt;br /&gt;&lt;br /&gt;As discussed in my previous blog post about declarative programming and deployment, declarativity is a spectrum -- the above specifications are (somewhat) declarative because they do not capture the activities to manage the life-cycle of the process (the &lt;strong&gt;how&lt;/strong&gt;). Instead, they specify &lt;strong&gt;what&lt;/strong&gt; process we want to run. The process manager derives and executes all activities to bring that process in a running state.&lt;br /&gt;&lt;br /&gt;sysvinit scripts themselves are not declarative, because they specify all activities (i.e. shell commands) that need to be executed to accomplish that goal. supervisord configurations and systemd services configuration files are (somewhat) declarative, because they capture process characteristics -- the process manager executes derives all required activities to bring the process in a running state.&lt;br /&gt;&lt;br /&gt;Despite the fact that I am not specifying any process management activities, these Nix expressions could still be considered somewhat a &quot;how specification&quot;, because each configuration is tailored towards a specific process manager. A process manager, such as syvinit, is a means to accomplish something else: getting a running process whose life-cycle can be conveniently managed.&lt;br /&gt;&lt;br /&gt;If I would revise the above specifications to only express what I kind of running process I want, disregarding the process manager, then I could simply write:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{createManagedProcess, nginx, stateDir}:&lt;br /&gt;{configFile, dependencies ? [], instanceSuffix ? &quot;&quot;}:&lt;br /&gt;&lt;br /&gt;let&lt;br /&gt;  instanceName = &quot;nginx${instanceSuffix}&quot;;&lt;br /&gt;  nginxLogDir = &quot;${stateDir}/${instanceName}/logs&quot;;&lt;br /&gt;in&lt;br /&gt;createManagedProcess {&lt;br /&gt;  name = instanceName;&lt;br /&gt;  description = &quot;Nginx&quot;;&lt;br /&gt;  initialize = ''&lt;br /&gt;    mkdir -p ${nginxLogDir}&lt;br /&gt;  '';&lt;br /&gt;  process = &quot;${nginx}/bin/nginx&quot;;&lt;br /&gt;  args = [ &quot;-c&quot; configFile&quot; -p&quot; &quot;${stateDir}/${instanceName}&quot; ];&lt;br /&gt;&lt;br /&gt;  inherit dependencies instanceName;&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;The above Nix expression simply states that we want to run a managed Nginx process (using certain command-line arguments) and before starting the process, we want to initialize the state by creating the log directory, if it does not exists yet.&lt;br /&gt;&lt;br /&gt;I can translate the above specification to all kinds of configuration artifacts that can be used by a variety of process managers to accomplish the same outcome. I have developed six kinds of generators allowing me to target the following process managers:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;sysvinit scripts, also known as &lt;a href=&quot;https://wiki.debian.org/LSBInitScripts&quot;&gt;LSB Init compliant scripts&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;a href=&quot;http://supervisord.org&quot;&gt;supervisord&lt;/a&gt; programs&lt;/li&gt;&lt;li&gt;&lt;a href=&quot;https://www.freedesktop.org/wiki/Software/systemd&quot;&gt;systemd&lt;/a&gt; services&lt;/li&gt;&lt;li&gt;&lt;a href=&quot;https://www.launchd.info&quot;&gt;launchd&lt;/a&gt; services&lt;/li&gt;&lt;li&gt;&lt;a href=&quot;https://www.freebsd.org/doc/en_US.ISO8859-1/articles/rc-scripting/index.html&quot;&gt;BSD rc&lt;/a&gt; scripts&lt;/li&gt;&lt;li&gt;Windows services (via Cygwin's &lt;a href=&quot;http://web.mit.edu/cygwin/cygwin_v1.3.2/usr/doc/Cygwin/cygrunsrv.README&quot;&gt;cygrunsrv&lt;/a&gt;)&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;Translating the properties of the process manager-agnostic configuration to a process manager-specific properties is quite straight forward for most concepts -- in many cases, there is a direct mapping between a property in the process manager-agnostic configuration to a process manager-specific property.&lt;br /&gt;&lt;br /&gt;For example, when we intend to target supervisord, then we can translate the &lt;i&gt;process&lt;/i&gt; and &lt;i&gt;args&lt;/i&gt; parameters to a &lt;i&gt;command&lt;/i&gt; invocation. For systemd, we can translate &lt;i&gt;process&lt;/i&gt; and &lt;i&gt;args&lt;/i&gt; to the &lt;i&gt;ExecStart&lt;/i&gt; property that refers to a command-line instruction that starts the process.&lt;br /&gt;&lt;br /&gt;Although the process manager-agnostic abstraction function supports enough features to get some well known system services working (e.g. Nginx, Apache HTTP service, PostgreSQL, MySQL etc.), it does not facilitate all possible features of each process manager -- it will provide a reasonable set of common features to get a process running and to impose some restrictions on it.&lt;br /&gt;&lt;br /&gt;It is still possible work around the feature limitations of process manager-agnostic deployment specifications. We can also influence the generation process by defining &lt;strong&gt;overrides&lt;/strong&gt; to get process manager-specific properties supported:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{createManagedProcess, nginx, stateDir}:&lt;br /&gt;{configFile, dependencies ? [], instanceSuffix ? &quot;&quot;}:&lt;br /&gt;&lt;br /&gt;let&lt;br /&gt;  instanceName = &quot;nginx${instanceSuffix}&quot;;&lt;br /&gt;  nginxLogDir = &quot;${stateDir}/${instanceName}/logs&quot;;&lt;br /&gt;in&lt;br /&gt;createManagedProcess {&lt;br /&gt;  name = instanceName;&lt;br /&gt;  description = &quot;Nginx&quot;;&lt;br /&gt;  initialize = ''&lt;br /&gt;    mkdir -p ${nginxLogDir}&lt;br /&gt;  '';&lt;br /&gt;  process = &quot;${nginx}/bin/nginx&quot;;&lt;br /&gt;  args = [ &quot;-c&quot; configFile&quot; -p&quot; &quot;${stateDir}/${instanceName}&quot; ];&lt;br /&gt;&lt;br /&gt;  inherit dependencies instanceName;&lt;br /&gt;&lt;br /&gt;  overrides = {&lt;br /&gt;    sysvinit = {&lt;br /&gt;      runlevels = [ 3 4 5 ];&lt;br /&gt;    };&lt;br /&gt;  };&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;In the above example, we have added an override specifically for sysvinit to tell that the init system that the process should be started in runlevels 3, 4 and 5 (which implies the process should stopped in the remaining runlevels: 0, 1, 2, and 6). The other process managers that I have worked with do not have a notion of runlevels.&lt;br /&gt;&lt;br /&gt;Similarly, we can use an override to, for example, use systemd-specific features to run a process in a Linux namespace etc.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;Simulating process manager-agnostic concepts with no direct equivalents&lt;/h2&gt;&lt;br /&gt;For some process manager-agnostic concepts, process managers do not always have direct equivalents. In such cases, there is still the possibility to apply non-trivial simulation strategies.&lt;br /&gt;&lt;br /&gt;&lt;h3&gt;Foreground processes or daemons&lt;/h3&gt;&lt;br /&gt;What all deployment specifications shown in this blog post have in common is that their main objective is to bring a process in a running state. How these processes are expected to behave is different among process managers.&lt;br /&gt;&lt;br /&gt;sysvinit and BSD rc scripts expect processes to &lt;strong&gt;daemonize&lt;/strong&gt; -- on invocation, a process spawns another process that keeps running in the background (the daemon process). After the initialization of the daemon process is done, the parent process terminates. If processes do not deamonize, the startup process execution blocks indefinitely.&lt;br /&gt;&lt;br /&gt;Daemons introduce another complexity from a process management perspective -- when invoking an executable from a shell session in background mode, the shell can you tell its process ID, so that it can be stopped when it is no longer necessary.&lt;br /&gt;&lt;br /&gt;With deamons, an invoked processes forks another child process (or when it supposed to really behave well: it double forks) that becomes the daemon process. The daemon process gets adopted by the init system, and thus remains in the background even if the shell session ends.&lt;br /&gt;&lt;br /&gt;The shell that invokes the executable does not know the PIDs of the resulting daemon processes, because that value is only propagated to the daemon's parent process, not the calling shell session. To still be able to control it, a well-behaving daemon typically writes its process IDs to a so-called PID file, so that it can be reliably terminated by a shell command when it is no longer required.&lt;br /&gt;&lt;br /&gt;sysvinit and BSD rc scripts extensively use PID files to control daemons. By using a process' PID file, the managing sysvinit/BSD rc script can tell you whether a process is running or not and reliably terminate a process instance.&lt;br /&gt;&lt;br /&gt;&quot;More modern&quot; process managers, such as launchd, supervisord, and cygrunsrv, do not work with processes that daemonize -- instead, these process managers are daemons themselves that invoke processes that work in &quot;foreground mode&quot;.&lt;br /&gt;&lt;br /&gt;One of the advantages of this approach is that services can be more reliably controlled -- because their PIDs are directly propagated to the controlling daemon from the &lt;i&gt;fork()&lt;/i&gt; library call, it is no longer required to work with PID files, that may not always work reliably (for example: a process might abrubtly terminate and never clean its PID file, giving the system the false impression that it is still running).&lt;br /&gt;&lt;br /&gt;systemd improves process control even further by using Linux cgroups -- although foreground process may be controlled more reliably than daemons, they can still fork other processes (e.g. a web service that creates processes per connection). When the controlling parent process terminates, and does not properly terminate its own child processes, they may keep running in the background indefintely. With cgroups it is possible for the process manager to retain control over all processes spawned by a service and terminate them when a service is no longer needed.&lt;br /&gt;&lt;br /&gt;systemd has another unique advantage over the other process managers -- it can work both with foreground processes and daemons, although foreground processes seem to have to preference according to the documentation, because they are much easier to control and develop.&lt;br /&gt;&lt;br /&gt;Many common system services, such as OpenSSH, MySQL or Nginx, have the ability to both run as a foreground process and as a daemon, typically by providing a command-line parameter or defining a property in a configuration file.&lt;br /&gt;&lt;br /&gt;To provide an optimal user experience for all supported process managers, it is typically a good thing in the process manager-agnostic deployment specification to specify both how a process can be used as a foreground process and as a daemon:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{createManagedProcess, nginx, stateDir, runtimeDir}:&lt;br /&gt;{configFile, dependencies ? [], instanceSuffix ? &quot;&quot;}:&lt;br /&gt;&lt;br /&gt;let&lt;br /&gt;  instanceName = &quot;nginx${instanceSuffix}&quot;;&lt;br /&gt;  nginxLogDir = &quot;${stateDir}/${instanceName}/logs&quot;;&lt;br /&gt;in&lt;br /&gt;createManagedProcess {&lt;br /&gt;  name = instanceName;&lt;br /&gt;  description = &quot;Nginx&quot;;&lt;br /&gt;  initialize = ''&lt;br /&gt;    mkdir -p ${nginxLogDir}&lt;br /&gt;  '';&lt;br /&gt;  process = &quot;${nginx}/bin/nginx&quot;;&lt;br /&gt;  args = [ &quot;-p&quot; &quot;${stateDir}/${instanceName}&quot; &quot;-c&quot; configFile ];&lt;br /&gt;  foregroundProcessExtraArgs = [ &quot;-g&quot; &quot;daemon off;&quot; ];&lt;br /&gt;  daemonExtraArgs = [ &quot;-g&quot; &quot;pid ${runtimeDir}/${instanceName}.pid;&quot; ];&lt;br /&gt;&lt;br /&gt;  inherit dependencies instanceName;&lt;br /&gt;&lt;br /&gt;  overrides = {&lt;br /&gt;    sysvinit = {&lt;br /&gt;      runlevels = [ 3 4 5 ];&lt;br /&gt;    };&lt;br /&gt;  };&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;In the above example, we have revised Nginx expression to both specify how the process can be started as a foreground process and as a daemon. The only thing that needs to be configured differently is one global directive in the Nginx configuration file -- by default, Nginx runs as a deamon, but by adding the &lt;i&gt;daemon off;&lt;/i&gt; directive to the configuration we can run it in foreground mode.&lt;br /&gt;&lt;br /&gt;When we run Nginx as daemon, we configure a PID file that refers to the instance name so that multiple instances can co-exist.&lt;br /&gt;&lt;br /&gt;To make this conveniently configurable, the above expression does the following:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;The &lt;i&gt;process&lt;/i&gt; parameter specifies the process that needs to be started both in foreground mode and as a daemon. The &lt;i&gt;args&lt;/i&gt; parameter specifies common command-line arguments that both the foreground and daemon process will use.&lt;/li&gt;&lt;li&gt;The &lt;i&gt;foregroundProcessExtraArgs&lt;/i&gt; parameter specifies additional command-line arguments that are only used when the process is started in foreground mode. In the above example, it is used to provide Nginx the global directive that disables the daemon setting.&lt;/li&gt;&lt;li&gt;The &lt;i&gt;daemonExtraArgs&lt;/i&gt; parameter specifies additional command-line arguments that are only used when the process is started as a daemon. In the above example, it used to provide Nginx a global directive with a PID file path that uniquely identifies the process instance.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;For custom software and services implemented in different language than C, e.g. Node.js, Java or Python, it is far less common that they have the ability to daemonize -- they can typically only be used as foreground processes.&lt;br /&gt;&lt;br /&gt;Nonetheless, we can still daemonize foreground-only processes, by using an external tool, such as &lt;a href=&quot;http://www.libslack.org/daemon/&quot;&gt;libslack's &lt;i&gt;daemon&lt;/i&gt;&lt;/a&gt; command:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$ daemon -U -i myforegroundprocess&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;The above command deamonizes the foreground process and creates a PID file for it, so that it can be managed by the sysvinit/BSD rc utility scripts.&lt;br /&gt;&lt;br /&gt;The opposite kind of &quot;simulation&quot; is also possible -- if a process can only be used as a daemon, then we can use a &lt;strong&gt;proxy process&lt;/strong&gt; to make it appear as a foreground process:&lt;br /&gt;&lt;br /&gt;&lt;pre style=&quot;overflow: auto;&quot;&gt;&lt;br /&gt;export _TOP_PID=$$&lt;br /&gt;&lt;br /&gt;# Handle to SIGTERM and SIGINT signals and forward them to the daemon process&lt;br /&gt;_term()&lt;br /&gt;{&lt;br /&gt;    trap &quot;exit 0&quot; TERM&lt;br /&gt;    kill -TERM &quot;$pid&quot;&lt;br /&gt;    kill $_TOP_PID&lt;br /&gt;}&lt;br /&gt;&lt;br /&gt;_interrupt()&lt;br /&gt;{&lt;br /&gt;    kill -INT &quot;$pid&quot;&lt;br /&gt;}&lt;br /&gt;&lt;br /&gt;trap _term SIGTERM&lt;br /&gt;trap _interrupt SIGINT&lt;br /&gt;&lt;br /&gt;# Start process in the background as a daemon&lt;br /&gt;${executable} &quot;$@&quot;&lt;br /&gt;&lt;br /&gt;# Wait for the PID file to become available.&lt;br /&gt;# Useful to work with daemons that don't behave well enough.&lt;br /&gt;count=0&lt;br /&gt;&lt;br /&gt;while [ ! -f &quot;${_pidFile}&quot; ]&lt;br /&gt;do&lt;br /&gt;    if [ $count -eq 10 ]&lt;br /&gt;    then&lt;br /&gt;        echo &quot;It does not seem that there isn't any pid file! Giving up!&quot;&lt;br /&gt;        exit 1&lt;br /&gt;    fi&lt;br /&gt;&lt;br /&gt;    echo &quot;Waiting for ${_pidFile} to become available...&quot;&lt;br /&gt;    sleep 1&lt;br /&gt;&lt;br /&gt;    ((count=count++))&lt;br /&gt;done&lt;br /&gt;&lt;br /&gt;# Determine the daemon's PID by using the PID file&lt;br /&gt;pid=$(cat ${_pidFile})&lt;br /&gt;&lt;br /&gt;# Wait in the background for the PID to terminate&lt;br /&gt;${if stdenv.isDarwin then ''&lt;br /&gt;  lsof -p $pid +r 3 &amp;amp;&amp;gt;/dev/null &amp;amp;&lt;br /&gt;'' else if stdenv.isLinux || stdenv.isCygwin then ''&lt;br /&gt;  tail --pid=$pid -f /dev/null &amp;amp;&lt;br /&gt; '' else if stdenv.isBSD || stdenv.isSunOS then ''&lt;br /&gt;   pwait $pid &amp;amp;&lt;br /&gt; '' else&lt;br /&gt;   throw &quot;Don't know how to wait for process completion on system: ${stdenv.system}&quot;}&lt;br /&gt;&lt;br /&gt;# Wait for the blocker process to complete.&lt;br /&gt;# We use wait, so that bash can still&lt;br /&gt;# handle the SIGTERM and SIGINT signals that may be sent to it by&lt;br /&gt;# a process manager&lt;br /&gt;blocker_pid=$!&lt;br /&gt;wait $blocker_pid&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;The idea of the proxy script shown above is that it runs as a foreground process as long as the daemon process is running and relays any relevant incoming signals (e.g. a terminate and interrupt) to the daemon process.&lt;br /&gt;&lt;br /&gt;Implementing this proxy was a bit tricky:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;In the beginning of the script we configure signal handlers for the &lt;i&gt;TERM&lt;/i&gt; and &lt;i&gt;INT&lt;/i&gt; signals so that the process manager can terminate the daemon process.&lt;/li&gt;&lt;li&gt;We must start the daemon and wait for it to become available. Although the parent process of a well-behaving daemon should only terminate when the initialization is done, this turns out not be a hard guarantee -- to make the process a bit more robust, we deliberately wait for the PID file to become available, before we attempt to wait for the termination of the daemon.&lt;/li&gt;&lt;li&gt;Then we wait for the PID to terminate. The bash shell has an internal &lt;i&gt;wait&lt;/i&gt; command that can be used to wait for a background process to terminate, but this only works with processes in the same process group as the shell. Daemons are in a new session (with different process groups), so they cannot be monitored by the shell by using the &lt;i&gt;wait&lt;/i&gt; command.&lt;br /&gt;&lt;br /&gt;&lt;a href=&quot;https://stackoverflow.com/questions/1058047/wait-for-a-process-to-finish&quot;&gt;From this Stackoverflow article&lt;/a&gt;, I learned that we can use the &lt;i&gt;tail&lt;/i&gt; command of GNU Coreutils, or &lt;i&gt;lsof&lt;/i&gt; on macOS/Darwin, and &lt;i&gt;pwait&lt;/i&gt; on BSDs and Solaris/SunOS to monitor processes in other process groups.&lt;/li&gt;&lt;li&gt;When a command is being executed by a shell script (e.g. in this particular case: &lt;i&gt;tail&lt;/i&gt;, &lt;i&gt;lsof&lt;/i&gt; or &lt;i&gt;pwait&lt;/i&gt;), the shell script can no longer respond to signals until the command completes. To still allow the script to respond to signals while it is waiting for the daemon process to terminate, we must run the previous command in background mode, and we use the &lt;i&gt;wait&lt;/i&gt; instruction to block the script. &lt;a href=&quot;https://unix.stackexchange.com/questions/146756/forward-sigterm-to-child-in-bash&quot;&gt;While a &lt;i&gt;wait&lt;/i&gt; command is running, the shell can respond to signals&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;The generator function will automatically pick the best solution for the selected target process manager -- this means that when our target process manager are sysvinit or BSD rc scripts, the generator automatically picks the configuration settings to run the process as a daemon. For the remaining process managers, the generator will pick the configuration settings that runs it as a foreground process.&lt;br /&gt;&lt;br /&gt;If a desired process model is not supported, then the generator will automatically simulate it. For instance, if we have a foreground-only process specification, then the generator will automatically configure a sysvinit script to call the &lt;i&gt;daemon&lt;/i&gt; executable to daemonize it.&lt;br /&gt;&lt;br /&gt;A similar process happens when a daemon-only process specification is deployed for a process manager that cannot work with it, such as supervisord.&lt;br /&gt;&lt;br /&gt;&lt;h3&gt;State initialization&lt;/h3&gt;&lt;br /&gt;Another important aspect in process deployment is &lt;strong&gt;state initialization&lt;/strong&gt;. Most system services require the presence of state directories in which they can store their PID, log and temp files. If these directories do not exist, the service may not work and refuse to start.&lt;br /&gt;&lt;br /&gt;To cope with this problem, I typically make processes self initializing -- before starting the process, I check whether the state has been intialized (e.g. check if the state directories exist) and re-initialize the initial state if needed.&lt;br /&gt;&lt;br /&gt;With most process managers, state initialization is easy to facilitate. For sysvinit and BSD rc scripts, we just use the generator to first execute the shell commands to initialize the state before the process gets started.&lt;br /&gt;&lt;br /&gt;Supervisord allows you to execute multiple shell commands in a single &lt;i&gt;command&lt;/i&gt; directive -- we can just execute a script that initializes the state before we execute the process that we want to manage.&lt;br /&gt;&lt;br /&gt;systemd has a &lt;i&gt;ExecStartPre&lt;/i&gt; directive that can be used to specify shell commands to execute before the main process starts.&lt;br /&gt;&lt;br /&gt;Apple launchd and cygrunsrv, however, do not have a generic shell execution mechanism or some facility allowing you to execute things before a process starts. Nonetheless, we can still ensure that the state is going to be initialized by creating a &lt;strong&gt;wrapper script&lt;/strong&gt; -- first the wrapper script does the state initialization and then executes the main process.&lt;br /&gt;&lt;br /&gt;If a state initialization procedure was specified and the target process manager does not support scripting, then the generator function will transparently wrap the main process into a wrapper script that supports state initialization.&lt;br /&gt;&lt;br /&gt;&lt;h3&gt;Process dependencies&lt;/h3&gt;&lt;br /&gt;Another important generic concept is process dependency management. For example, Nginx can act as a reverse proxy for another web application process. To provide a functional Nginx service, we must be sure that the web application process gets activated as well, and that the web application is activated before Nginx.&lt;br /&gt;&lt;br /&gt;If the web application process is activated after Nginx or missing completely, then Nginx is (temporarily) unable to redirect incoming requests to the web application process causing end-users to see bad gateway errors.&lt;br /&gt;&lt;br /&gt;The process managers that I have experimented with all have a different notion of process dependencies.&lt;br /&gt;&lt;br /&gt;sysvinit scripts can optionally declare dependencies in their comment sections. Tools that know how to interpret these dependency specifications can use it to decide the right activation order. Systems using sysvinit typically ignore this specification. Instead, they work with sequence numbers in their file names -- each run level configuration directory contains a prefix (S or K) followed by two numeric digits that defines the start or stop order.&lt;br /&gt;&lt;br /&gt;supervisord does not work with dependency specifications, but every program can optionally provide a &lt;i&gt;priority&lt;/i&gt; setting that can be used to order the activation and deactivation of programs -- lower priority numbers have precedence over high priority numbers.&lt;br /&gt;&lt;br /&gt;From dependency specifications in a process management expression, the generator function can automatically derive sequence numbers for process managers that require it.&lt;br /&gt;&lt;br /&gt;Similar to sysvinit scripts, BSD rc scripts can also declare dependencies in their comment sections. Contrary to sysvinit scripts, BSD rc scripts can use the &lt;a href=&quot;https://www.freebsd.org/cgi/man.cgi?rcorder(8)&quot;&gt;&lt;i&gt;rcorder&lt;/i&gt;&lt;/a&gt; tool to parse these dependencies from the comments section and automatically derive the order in which the BSD rc scripts need to be activated.&lt;br /&gt;&lt;br /&gt;&lt;i&gt;cygrunsrv&lt;/i&gt; also allows you directly specify process dependencies. The Windows service manager makes sure that the service get activated in the right order and that all process dependencies are activated first. The only limitation is that cygrunsrv only allows up to 16 dependencies to be specified per service.&lt;br /&gt;&lt;br /&gt;To simulate process dependencies with systemd, we can use two properties. The &lt;i&gt;Wants&lt;/i&gt; property can be used to tell systemd that another service needs to be activated first. The &lt;i&gt;After&lt;/i&gt; property can be used to specify the ordering.&lt;br /&gt;&lt;br /&gt;Sadly, it seems that launchd has no notion of process dependencies at all -- processes can be activated by certain events, e.g. when a kernel module was loaded or through socket activation, but it does not seem to have the ability to configure process dependencies or the activation ordering. When our target process manager is launchd, then we simply have to inform the user that proper activation ordering cannot be guaranteed.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;Changing user privileges&lt;/h2&gt;&lt;br /&gt;Another general concept, that has subtle differences in each process manager, is changing user privileges. Typically for the deployment of system services, you do not want to run these services as root user (that has full access to the filesystem), but as an unprivileged user.&lt;br /&gt;&lt;br /&gt;sysvinit and BSD rc scripts have to change users through the &lt;i&gt;su&lt;/i&gt; command. The &lt;i&gt;su&lt;/i&gt; command can be used to change the user ID (UID), and will automatically adopt the primary group ID (GID) of the corresponding user.&lt;br /&gt;&lt;br /&gt;Supervisord and &lt;i&gt;cygrunsrv&lt;/i&gt; can also only change user IDs (UIDs), and will adopt the primary group ID (GID) of the corresponding user.&lt;br /&gt;&lt;br /&gt;Systemd and launchd can both change the user IDs and group IDs of the process that it invokes.&lt;br /&gt;&lt;br /&gt;Because only changing UIDs are universally supported amongst process managers, I did not add a configuration property that allows you to change GIDs in a process manager-agnostic way.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;Deploying process manager-agnostic configurations&lt;/h2&gt;&lt;br /&gt;With a processes Nix expression, we can define which process instances we want to run (and how they can be constructed from source code and their dependencies):&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{ pkgs ? import  { inherit system; }&lt;br /&gt;, system ? builtins.currentSystem&lt;br /&gt;, stateDir ? &quot;/var&quot;&lt;br /&gt;, runtimeDir ? &quot;${stateDir}/run&quot;&lt;br /&gt;, logDir ? &quot;${stateDir}/log&quot;&lt;br /&gt;, tmpDir ? (if stateDir == &quot;/var&quot; then &quot;/tmp&quot; else &quot;${stateDir}/tmp&quot;)&lt;br /&gt;, forceDisableUserChange ? false&lt;br /&gt;, processManager&lt;br /&gt;}:&lt;br /&gt;&lt;br /&gt;let&lt;br /&gt;  constructors = import ./constructors.nix {&lt;br /&gt;    inherit pkgs stateDir runtimeDir logDir tmpDir;&lt;br /&gt;    inherit forceDisableUserChange processManager;&lt;br /&gt;  };                                                                                                                                                                                               &lt;br /&gt;in                                                                                                                                                                                                 &lt;br /&gt;rec {                                                                                                                                                                                              &lt;br /&gt;  webapp = rec {                                                                                                                                                                                   &lt;br /&gt;    port = 5000;                                                                                                                                                                                   &lt;br /&gt;    dnsName = &quot;webapp.local&quot;;                                                                                                                                                                      &lt;br /&gt;                                                                                                                                                                                                   &lt;br /&gt;    pkg = constructors.webapp {                                                                                                                                                                    &lt;br /&gt;      inherit port;                                                                                                                                                                                &lt;br /&gt;    };                                                                                                                                                                                             &lt;br /&gt;  };                                                                                                                                                                                               &lt;br /&gt;                                                                                                                                                                                                   &lt;br /&gt;  nginxReverseProxy = rec {&lt;br /&gt;    port = 8080;&lt;br /&gt;&lt;br /&gt;    pkg = constructors.nginxReverseProxy {&lt;br /&gt;      webapps = [ webapp ];&lt;br /&gt;      inherit port;&lt;br /&gt;    } {};&lt;br /&gt;  };&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;In the above Nix expression, we compose two running process instances:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;&lt;i&gt;webapp&lt;/i&gt; is a trivial web application process that will simply return a static HTML page by using the HTTP protocol.&lt;/li&gt;&lt;li&gt;&lt;i&gt;nginxReverseProxy&lt;/i&gt; is a Nginx server configured as a reverse proxy server. It will forward incoming HTTP requests to the appropriate web application instance, based on the virtual host name. If a virtual host name is &lt;i&gt;webapp.local&lt;/i&gt;, then Nginx forwards the request to the &lt;i&gt;webapp&lt;/i&gt; instance.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;To generate the configuration artifacts for the process instances, we refer to a separate constructors Nix expression. Each constructor will call the &lt;i&gt;createManagedProcess&lt;/i&gt; function abstraction (as shown earlier) to construct a process configuration in a process manager-agnostic way.&lt;br /&gt;&lt;br /&gt;With the following command-line instruction, we can generate sysvinit scripts for the &lt;i&gt;webapp&lt;/i&gt; and Nginx processes declared in the processes expression, and run them as an unprivileged user with the state files managed in our home directory:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$ nixproc-build --process-manager sysvinit \&lt;br /&gt;  --state-dir /home/sander/var \&lt;br /&gt;  --force-disable-user-change processes.nix&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;By adjusting the &lt;i&gt;--process-manager&lt;/i&gt; parameter we can also generate artefacts for a different process manager. For example, the following command will generate systemd unit config files instead of sysvinit scripts:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$ nixproc-build --process-manager systemd \&lt;br /&gt;  --state-dir /home/sander/var \&lt;br /&gt;  --force-disable-user-change processes.nix&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;The following command will automatically build and deploy all processes, using sysvinit as a process manager:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$ nixproc-sysvinit-switch --state-dir /home/sander/var \&lt;br /&gt;  --force-disable-user-change processes.nix&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;We can also run a life-cycle management activity on all previously deployed processes. For example, to retrieve the statuses of all processes, we can run:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$ nixproc-sysvinit-runactivity status&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;We can also traverse the processes in reverse dependency order. This is particularly useful to reliably stop all processes, without breaking any process dependencies:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$ nixproc-sysvinit-runactivity -r stop&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Similarly, there are command-line tools to use the other supported process managers. For example, to deploy systemd units instead of sysvinit scripts, you can run:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$ nixproc-systemd-switch processes.nix&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;&lt;h2&gt;Distributed process manager-agnostic deployment with Disnix&lt;/h2&gt;&lt;br /&gt;As shown in the previous process management framework blog post, it is also possible to deploy processes to machines in a network and have inter-dependencies between processes. These kinds of deployments can be managed by &lt;a href=&quot;https://sandervanderburg.blogspot.com/2011/02/disnix-toolset-for-distributed.html&quot;&gt;Disnix&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;Compared to the previous blog post (in which we could only deploy sysvinit scripts), we can now also use any process manager that the framework supports. The Dysnomia toolset provides plugins that supports all process managers that this framework supports:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{ pkgs, distribution, invDistribution, system&lt;br /&gt;, stateDir ? &quot;/var&quot;&lt;br /&gt;, runtimeDir ? &quot;${stateDir}/run&quot;&lt;br /&gt;, logDir ? &quot;${stateDir}/log&quot;&lt;br /&gt;, tmpDir ? (if stateDir == &quot;/var&quot; then &quot;/tmp&quot; else &quot;${stateDir}/tmp&quot;)&lt;br /&gt;, forceDisableUserChange ? false&lt;br /&gt;, processManager ? &quot;sysvinit&quot;&lt;br /&gt;}:&lt;br /&gt;&lt;br /&gt;let&lt;br /&gt;  constructors = import ./constructors.nix {&lt;br /&gt;    inherit pkgs stateDir runtimeDir logDir tmpDir;&lt;br /&gt;    inherit forceDisableUserChange processManager;&lt;br /&gt;  };&lt;br /&gt;&lt;br /&gt;  processType =&lt;br /&gt;    if processManager == &quot;sysvinit&quot; then &quot;sysvinit-script&quot;&lt;br /&gt;    else if processManager == &quot;systemd&quot; then &quot;systemd-unit&quot;&lt;br /&gt;    else if processManager == &quot;supervisord&quot; then &quot;supervisord-program&quot;&lt;br /&gt;    else if processManager == &quot;bsdrc&quot; then &quot;bsdrc-script&quot;&lt;br /&gt;    else if processManager == &quot;cygrunsrv&quot; then &quot;cygrunsrv-service&quot;&lt;br /&gt;    else throw &quot;Unknown process manager: ${processManager}&quot;;&lt;br /&gt;in&lt;br /&gt;rec {&lt;br /&gt;  webapp = rec {&lt;br /&gt;    name = &quot;webapp&quot;;&lt;br /&gt;    port = 5000;&lt;br /&gt;    dnsName = &quot;webapp.local&quot;;&lt;br /&gt;    pkg = constructors.webapp {&lt;br /&gt;      inherit port;&lt;br /&gt;    };&lt;br /&gt;    type = processType;&lt;br /&gt;  };&lt;br /&gt;&lt;br /&gt;  nginxReverseProxy = rec {&lt;br /&gt;    name = &quot;nginxReverseProxy&quot;;&lt;br /&gt;    port = 8080;&lt;br /&gt;    pkg = constructors.nginxReverseProxy {&lt;br /&gt;      inherit port;&lt;br /&gt;    };&lt;br /&gt;    dependsOn = {&lt;br /&gt;      inherit webapp;&lt;br /&gt;    };&lt;br /&gt;    type = processType;&lt;br /&gt;  };&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;In the above expression, we have extended the previously shown processes expression into a Disnix service expression, in which every attribute in the attribute set represents a service that can be distributed to a target machine in the network.&lt;br /&gt;&lt;br /&gt;The &lt;i&gt;type&lt;/i&gt; attribute of each service indicates which Dysnomia plugin needs to manage its life-cycle. We can automatically select the appropriate plugin for our desired process manager by deriving it from the &lt;i&gt;processManager&lt;/i&gt; parameter.&lt;br /&gt;&lt;br /&gt;The above Disnix expression has a drawback -- in a &lt;strong&gt;heteregenous network&lt;/strong&gt; of machines (that run multiple operating systems and/or process managers), we need to compose all desired variants of each service with configuration files for each process manager that we want to use.&lt;br /&gt;&lt;br /&gt;It is also possible to have &lt;strong&gt;target-agnostic&lt;/strong&gt; services, by delegating the translation steps to the corresponding target machines. Instead of directly generating a configuration file for a process manager, we generate a JSON specification containing all parameters that are passed to &lt;i&gt;createManagedProcess&lt;/i&gt;. We can use this JSON file to build the corresponding configuration artefacts on the target machine:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{ pkgs, distribution, invDistribution, system&lt;br /&gt;, stateDir ? &quot;/var&quot;&lt;br /&gt;, runtimeDir ? &quot;${stateDir}/run&quot;&lt;br /&gt;, logDir ? &quot;${stateDir}/log&quot;&lt;br /&gt;, tmpDir ? (if stateDir == &quot;/var&quot; then &quot;/tmp&quot; else &quot;${stateDir}/tmp&quot;)&lt;br /&gt;, forceDisableUserChange ? false&lt;br /&gt;, processManager ? null&lt;br /&gt;}:&lt;br /&gt;&lt;br /&gt;let&lt;br /&gt;  constructors = import ./constructors.nix {&lt;br /&gt;    inherit pkgs stateDir runtimeDir logDir tmpDir;&lt;br /&gt;    inherit forceDisableUserChange processManager;&lt;br /&gt;  };&lt;br /&gt;in&lt;br /&gt;rec {&lt;br /&gt;  webapp = rec {&lt;br /&gt;    name = &quot;webapp&quot;;&lt;br /&gt;    port = 5000;&lt;br /&gt;    dnsName = &quot;webapp.local&quot;;&lt;br /&gt;    pkg = constructors.webapp {&lt;br /&gt;      inherit port;&lt;br /&gt;    };&lt;br /&gt;    type = &quot;managed-process&quot;;&lt;br /&gt;  };&lt;br /&gt;&lt;br /&gt;  nginxReverseProxy = rec {&lt;br /&gt;    name = &quot;nginxReverseProxy&quot;;&lt;br /&gt;    port = 8080;&lt;br /&gt;    pkg = constructors.nginxReverseProxy {&lt;br /&gt;      inherit port;&lt;br /&gt;    };&lt;br /&gt;    dependsOn = {&lt;br /&gt;      inherit webapp;&lt;br /&gt;    };&lt;br /&gt;    type = &quot;managed-process&quot;;&lt;br /&gt;  };&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;In the above services model, we have set the &lt;i&gt;processManager&lt;/i&gt; parameter to &lt;i&gt;null&lt;/i&gt; causing the generator to print JSON presentations of the function parameters passed to &lt;i&gt;createManagedProcess&lt;/i&gt;.&lt;br /&gt;&lt;br /&gt;The &lt;i&gt;managed-process&lt;/i&gt; type refers to a Dysnomia plugin that consumes the JSON specification and invokes the &lt;i&gt;createManagedProcess&lt;/i&gt; function to convert the JSON configuration to a configuration file used by the preferred process manager.&lt;br /&gt;&lt;br /&gt;In the infrastructure model, we can configure the preferred process manager for each target machine:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{&lt;br /&gt;  test1 = {&lt;br /&gt;    properties = {&lt;br /&gt;      hostname = &quot;test1&quot;;&lt;br /&gt;    };&lt;br /&gt;    containers = {&lt;br /&gt;      managed-process = {&lt;br /&gt;        processManager = &quot;sysvinit&quot;;&lt;br /&gt;      };&lt;br /&gt;    };&lt;br /&gt;  };&lt;br /&gt;&lt;br /&gt;  test2 = {&lt;br /&gt;    properties = {&lt;br /&gt;      hostname = &quot;test2&quot;;&lt;br /&gt;    };&lt;br /&gt;    containers = {&lt;br /&gt;      managed-process = {&lt;br /&gt;        processManager = &quot;systemd&quot;;&lt;br /&gt;      };&lt;br /&gt;    };&lt;br /&gt;  };&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;In the above infrastructure model, the &lt;i&gt;managed-proces&lt;/i&gt; container on the first machine: &lt;i&gt;test1&lt;/i&gt; has been configured to use sysvinit scripts to manage processes. On the second test machine: &lt;i&gt;test2&lt;/i&gt; the &lt;i&gt;managed-process&lt;/i&gt; container is configured to use systemd to manage processes.&lt;br /&gt;&lt;br /&gt;If we distribute the services in the services model to targets in the infrastructure model as follows:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;{infrastructure}:&lt;br /&gt;&lt;br /&gt;{&lt;br /&gt;  webapp = [ infrastructure.test1 ];&lt;br /&gt;  nginxReverseProxy = [ infrastructure.test2 ];&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;and the deploy the system as follows:&lt;br /&gt;&lt;br /&gt;&lt;pre style=&quot;overflow: auto; font-size: 90%;&quot;&gt;&lt;br /&gt;$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Then the &lt;i&gt;webapp&lt;/i&gt; process will distributed to the &lt;i&gt;test1&lt;/i&gt; machine in the network and will be managed with a sysvinit script.&lt;br /&gt;&lt;br /&gt;The &lt;i&gt;nginxReverseProxy&lt;/i&gt; will be deployed to the &lt;i&gt;test2&lt;/i&gt; machine and managed as a systemd job. The Nginx reverse proxy forwards incoming connections to the &lt;i&gt;webapp.local&lt;/i&gt; domain name to the web application process hosted on the first machine.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;Discussion&lt;/h2&gt;&lt;br /&gt;In this blog post, I have introduced a process manager-agnostic function abstraction making it possible to target all kinds of process managers on a variety of operating systems.&lt;br /&gt;&lt;br /&gt;By using a single set of declarative specifications, we can:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Target six different process managers on four different kinds of operating systems.&lt;/li&gt;&lt;li&gt;Implement various kinds of deployment scenarios: production deployments, test deployments as an unprivileged user.&lt;/li&gt;&lt;li&gt;Construct multiple instances of processes.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;In a distributed-context, the advantage is that we can uniformly target all supported process managers and operating systems in a heterogeneous environment from a single declarative specification.&lt;br /&gt;&lt;br /&gt;This is particularly useful to facilitate technology diversity -- for example, one of the key selling points of Microservices is that &quot;any technology&quot; can be used to implement them. In many cases, technology diversity is &quot;restricted&quot; to frameworks, programming languages, and storage technologies.&lt;br /&gt;&lt;br /&gt;One particular aspect that is rarely changed is the choice of operating systems, because of the limitations of deployment tools -- most deployment solutions for Microservices are container-based and heavily rely on Linux-only concepts, such as Namespaces and cgroups.&lt;br /&gt;&lt;br /&gt;With this process managemenent framework and the recent Dysnomia plugin additions for Disnix, it is possible to target all kinds of operating systems that support the Nix package manager, making the operating system component selectable as well. This, for example, allows you to also pick the best operating system to implement a certain requirement -- for example, when performance is important you might pick Linux, and when there is a strong emphasis on security, you could pick OpenBSD to host a mission criticial component.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;Limitations&lt;/h2&gt;&lt;br /&gt;The following table, summarizes the differences between the process manager solutions that I have investigated:&lt;br /&gt;&lt;br /&gt;&lt;div&gt;&lt;table style=&quot;border-style: solid; border-width: 1px;&quot;&gt;&lt;tbody&gt;&lt;tr&gt;&lt;th&gt;&lt;/th&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;sysvinit&lt;/th&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;bsdrc&lt;/th&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;supervisord&lt;/th&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;systemd&lt;/th&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;launchd&lt;/th&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;cygrunsrv&lt;/th&gt;&lt;/tr&gt; &lt;tr&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Process type&lt;/th&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;daemon&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;daemon&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;foreground&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;foreground&lt;br /&gt;daemon&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;foreground&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;foreground&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Process control method&lt;/th&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;PID files&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;PID files&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Process PID&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;cgroups&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Process PID&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Process PID&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Scripting support&lt;/th&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;yes&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;yes&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;yes&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;yes&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;no&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;no&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Process dependency management&lt;/th&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Numeric ordering&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Dependency-based&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Numeric ordering&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Dependency-based&lt;br /&gt;+ dependency loading&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;None&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Dependency-based&lt;br /&gt;+ dependency loading&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;User changing capabilities&lt;/th&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;user&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;user&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;user and group&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;user and group&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;user and group&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;user&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Unprivileged user deployments&lt;/th&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;yes*&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;yes*&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;yes&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;yes*&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;no&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;no&lt;/td&gt;&lt;/tr&gt; &lt;tr&gt;&lt;th style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Operating system support&lt;/th&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Linux&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;FreeBSD&lt;br /&gt; &amp;gt;OpenBSD&lt;br /&gt;NetBSD&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Many UNIX-like:&lt;br /&gt;Linux&lt;br /&gt;macOS&lt;br /&gt;FreeBSD&lt;br /&gt;Solaris&lt;br /&gt;&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Linux (+glibc) only&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;macOS (Darwin)&lt;/td&gt;&lt;td style=&quot;border-style: solid; border-width: 1px; white-space: nowrap;&quot;&gt;Windows (Cygwin)&lt;/td&gt;&lt;/tr&gt; &lt;/tbody&gt;&lt;/table&gt;&lt;/div&gt;&lt;br /&gt;Although we can facilitate lifecycle management from a common specification with a variety of process managers, only the most important common features are supported.&lt;br /&gt;&lt;br /&gt;Not every concept can be done in a process manager agnostic way. For example, we cannot generically do any isolation of resources (except for packages, because we use Nix). It is difficult to generalize these concepts because these they are not standardized, e.g. the POSIX standard does not descibe namespaces and cgroups (or similar concepts).&lt;br /&gt;&lt;br /&gt;Furthermore, most process managers (with the exception of supervisord) are operating system specific. As a result, it still matters what process manager is picked.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;Related work&lt;/h2&gt;&lt;br /&gt;Process manager-agnostic deployment is not entirely a new idea. Dysnomia already has a target-agnostic 'process' plugin for quite a while, that translates a simple deployment specification (constisting of key-value pairs) to a systemd unit configuration file or sysvinit script.&lt;br /&gt;&lt;br /&gt;The features of Dysnomia's &lt;i&gt;process&lt;/i&gt; plugin are much more limited compared to the &lt;i&gt;createManagedProcess&lt;/i&gt; abstraction function described in this blog post. It does not support any other than process managers than sysvint and systemd, and it can only work with foreground processes.&lt;br /&gt;&lt;br /&gt;Furthermore, target agnostic configurations cannot be easily extended -- it is possible to (ab)use the templating mechanism, but it has no first class overridde facilities.&lt;br /&gt;&lt;br /&gt;I also found a project called &lt;a href=&quot;https://github.com/jordansissel/pleaserun&quot;&gt;pleaserun&lt;/a&gt; that also has the objective to generate configuration files for a variety of process managers (my approach and pleaserunit, both support sysvinit scripts, systemd and launchd).&lt;br /&gt;&lt;br /&gt;It seems to use template files to generate the configuration artefacts, and it does not seem to have a generic extension mechanism. Furthermore, it provides no framework to configure the location of shared resources, automatically install package dependencies or to compose multiple instances of processes.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;Some remaining thoughts&lt;/h2&gt;&lt;br /&gt;Although the Nix package manager (not the NixOS distribution), should be portable amongst a variety of UNIX-like systems, it turns out that the only two operating systems that are well supported are Linux and macOS. Nix was reported to work on a variety of other UNIX-like systems in the past, but recently it seems that many things are broken.&lt;br /&gt;&lt;br /&gt;To make Nix work on FreeBSD 12.1, I have used the latest stable Nix package manager version &lt;a href=&quot;https://github.com/0mp/freebsd-ports-nix&quot;&gt;with patches from this repository&lt;/a&gt;. It turns out that there is still a patch missing to work around in a bug in FreeBSD that incorrectly kills all processes in a process group. Fortunately, when we run Nix as as unprivileged user, this bug does not seem to cause any serious problems.&lt;br /&gt;&lt;br /&gt;Recent versions of Nixpkgs turn out to be horribly broken on FreeBSD -- the FreeBSD stdenv does not seem to work at all. I tried switching back to stdenv-native (a &lt;i&gt;stdenv&lt;/i&gt; environment that impurely uses the host system's compiler and core executables), but that also no longer seems to work in the last three major releases -- the Nix expression evaluation breaks in several places. Due to the intense amount of changes and assumptions that the &lt;i&gt;stdenv&lt;/i&gt; infrastructure currently makes, it was as good as impossible for me to fix the infrastructure.&lt;br /&gt;&lt;br /&gt;As another workaround, I reverted back very to a very old version of Nixpkgs (version 17.03 to be precise), that still has a working stdenv-native environment. With some tiny adjustments (e.g. adding some shell aliases for some GNU variants of certain shell executables to &lt;i&gt;stdenv-native&lt;/i&gt;), I have managed to get some basic Nix packages working, including Nginx on FreeBSD.&lt;br /&gt;&lt;br /&gt;Surprisingly, running Nix on Cygwin was less painful than FreeBSD (because of all the GNUisms that Cygwin provides). Similar to FreeBSD, recent versions of Nixpkgs also appear to be broken, including the Cygwin stdenv environment. By reverting back to &lt;i&gt;release-18.03&lt;/i&gt; (that still has a somewhat working &lt;i&gt;stdenv&lt;/i&gt; for Cygwin), I have managed to build a working Nginx version.&lt;br /&gt;&lt;br /&gt;As a future improvement to Nixpkgs, I would like to propose a testing solution for stdenv-native. Although I understand that is difficult to dedicate manpower to maintain all unconventional Nix/Nixpkgs ports, stdenv-native is something that we can also convienently test on Linux and prevent from breaking in the future.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;Availability&lt;/h2&gt;&lt;br /&gt;&lt;a href=&quot;https://github.com/svanderburg/nix-processmgmt&quot;&gt;The latest version of my experimental Nix-based process framework&lt;/a&gt;, that includes the process manager-agnostic configuration function described in this blog post, can be obtained from my GitHub page.&lt;br /&gt;&lt;br /&gt;In addition, the repository also contains some example cases, including the web application system described in this blog post, and a set of common system services: MySQL, Apache HTTP server, PostgreSQL and Apache Tomcat.&lt;br /&gt;&lt;br /&gt;</description>
	<pubDate>Sat, 15 Feb 2020 20:07:00 +0000</pubDate>
	<author>noreply@blogger.com (Sander van der Burg)</author>
</item>
<item>
	<title>Cachix: CDN and double storage size</title>
	<guid isPermaLink="true">https://blog.cachix.org/post/2020-01-28-cdn-and-double-storage/</guid>
	<link>https://blog.cachix.org/post/2020-01-28-cdn-and-double-storage/</link>
	<description>Cachix - Nix binary cache hosting, has grown quite a bit in recent months in terms of day to day usage and that was mostly noticable on bandwidth.
Over 3000 GB were served in December 2019.
CDN by CloudFlare Increased usage prompted a few backend machine instance upgrades to handle concurrent upload/downloads, but it became clear it’s time to abandon single machine infrastructure.
As of today, all binary caches are served by CloudFlare CDN.</description>
	<pubDate>Wed, 29 Jan 2020 08:00:00 +0000</pubDate>
	<author>support@cachix.org (Domen Kožar)</author>
</item>
<item>
	<title>Mayflower: __structuredAttrs in Nix</title>
	<guid isPermaLink="true">https://nixos.mayflower.consulting/blog/2020/01/20/structured-attrs/</guid>
	<link>https://nixos.mayflower.consulting/blog/2020/01/20/structured-attrs/</link>
	<description>In Nix 2 a new parameter to the derivation primitive was added. It changes how information is passed to the derivation builder.
Current State In order to show how it changes the handling of parameters to derivation, the first example will show the current state with __structuredAttrs set to false and the stdenv.mkDerivation wrapper around derivation. All parameters are passed to the builder as environment variables, canonicalised by Nix in imitation of shell script conventions:</description>
	<pubDate>Mon, 20 Jan 2020 12:00:00 +0000</pubDate>
</item>
<item>
	<title>Hercules Labs: Hercules CI &amp; Cachix split up</title>
	<guid isPermaLink="true">https://blog.hercules-ci.com/2020/01/14/hercules-ci-cachix-split-up/</guid>
	<link>https://blog.hercules-ci.com/2020/01/14/hercules-ci-cachix-split-up/</link>
	<description>&lt;p&gt;After careful consideration of how to balance between the two products, we’ve decided to split up. Each of the two products will be a separate entity:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Hercules CI becomes part of Robert Hensing’s Ensius B.V.&lt;/li&gt;
  &lt;li&gt;Cachix becomes part of Domen Kožar’s Enlambda OÜ&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For customers there will be no changes, except for the point of contact in support requests.&lt;/p&gt;

&lt;p&gt;Domen &amp;amp; Robert&lt;/p&gt;</description>
	<pubDate>Tue, 14 Jan 2020 00:00:00 +0000</pubDate>
</item>
<item>
	<title>Mayflower: Windows-on-NixOS, part 1: Migrating bare-metal to a VM</title>
	<guid isPermaLink="true">https://nixos.mayflower.consulting/blog/2019/11/27/windows-vm-storage/</guid>
	<link>https://nixos.mayflower.consulting/blog/2019/11/27/windows-vm-storage/</link>
	<description>This is part 1 of a series of blog posts explaining how we took an existing Windows installation on hardware and moved it into a VM running on top of NixOS.
Background We have a decently-equipped desktop PC sitting in our office, which is designated for data experiments using TensorFlow and such. During off-hours, it’s also used for games, and for that purpose it has Windows installed on it. We decided to try moving Windows into a VM within NixOS so that we could run both operating systems in parallel.</description>
	<pubDate>Wed, 27 Nov 2019 06:00:00 +0000</pubDate>
</item>
<item>
	<title>Craige McWhirter: Deploying and Configuring Vim on NixOS</title>
	<guid isPermaLink="true">http://mcwhirter.com.au//craige/blog/2019/Deploying_and_Configuring_Vim_on_NixOS/</guid>
	<link>http://mcwhirter.com.au//craige/blog/2019/Deploying_and_Configuring_Vim_on_NixOS/</link>
	<description>&lt;p&gt;&lt;img alt=&quot;NixOS Gears by Craige McWhirter&quot; src=&quot;http://mcwhirter.com.au/files/NixOS_Gears.png&quot; title=&quot;NixOS Gears by Craige McWhirter&quot; /&gt;&lt;/p&gt;

&lt;p&gt;I had a need to deploy &lt;a href=&quot;https://www.vim.org/&quot;&gt;vim&lt;/a&gt; and my particular preferred
configuration both system-wide and across multiple systems (via
&lt;a href=&quot;https://nixos.org/nixops/&quot;&gt;NixOps&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;I started by creating a file named &lt;code&gt;vim.nix&lt;/code&gt; that would be imported into either
&lt;code&gt;/etc/nixos/configuration.nix&lt;/code&gt; or an appropriate NixOps Nix file. This example
is a stub that shows a number of common configuration items:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://source.mcwhirter.io/craige/nixos-examples/src/branch/master/applications/editors/vim.nix&quot;&gt;vim.nix&lt;/a&gt;:&lt;/p&gt;

&lt;pre&gt;&lt;code class=&quot;nix&quot;&gt;with import &amp;lt;nixpkgs&amp;gt; {};

vim_configurable.customize {
  name = &quot;vim&quot;;   # Specifies the vim binary name.
  # Below you can specify what usually goes into `~/.vimrc`
  vimrcConfig.customRC = ''
    &quot; Preferred global default settings:
    set number                    &quot; Enable line numbers by default
    set background=dark           &quot; Set the default background to dark or light
    set smartindent               &quot; Automatically insert extra level of indentation
    set tabstop=4                 &quot; Default tabstop
    set shiftwidth=4              &quot; Default indent spacing
    set expandtab                 &quot; Expand [TABS] to spaces
    syntax enable                 &quot; Enable syntax highlighting
    colorscheme solarized         &quot; Set the default colour scheme
    set t_Co=256                  &quot; use 265 colors in vim
    set spell spelllang=en_au     &quot; Default spell checking language
    hi clear SpellBad             &quot; Clear any unwanted default settings
    hi SpellBad cterm=underline   &quot; Set the spell checking highlight style
    hi SpellBad ctermbg=NONE      &quot; Set the spell checking highlight background
    match ErrorMsg '\s\+$'        &quot;

    let g:airline_powerline_fonts = 1   &quot; Use powerline fonts
    let g:airline_theme='solarized'     &quot; Set the airline theme

    set laststatus=2   &quot; Set up the status line so it's coloured and always on

    &quot; Add more settings below
  '';
  # store your plugins in Vim packages
  vimrcConfig.packages.myVimPackage = with pkgs.vimPlugins; {
    start = [               # Plugins loaded on launch
      airline               # Lean &amp;amp; mean status/tabline for vim that's light as air
      solarized             # Solarized colours for Vim
      vim-airline-themes    # Collection of themes for airlin
      vim-nix               # Support for writing Nix expressions in vim
    ];
    # manually loadable by calling `:packadd $plugin-name`
    # opt = [ phpCompletion elm-vim ];
    # To automatically load a plugin when opening a filetype, add vimrc lines like:
    # autocmd FileType php :packadd phpCompletion
  };
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;I then needed to import this file into my system packages stanza:&lt;/p&gt;

&lt;pre&gt;&lt;code class=&quot;nix&quot;&gt;  environment = {
    systemPackages = with pkgs; [
      someOtherPackages   # Normal package listing
      (
        import ./vim.nix
      )
    ];
  };
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This will then install and configure Vim as you've defined it.&lt;/p&gt;

&lt;p&gt;If you'd like to give this build a run in a non-production space, I've written &lt;a href=&quot;https://source.mcwhirter.io/craige/nixos-examples/src/branch/master/applications/editors/vim_vm.nix&quot;&gt;vim_vm.nix&lt;/a&gt; with which you can build a VM, ssh into afterwards and test the Vim configuration:&lt;/p&gt;

&lt;pre&gt;&lt;code class=&quot;bash&quot;&gt;$ nix-build '&amp;lt;nixpkgs/nixos&amp;gt;' -A vm --arg configuration ./vim_vm.nix
...
$ export QEMU_OPTS=&quot;-m 4192&quot;
$ export QEMU_NET_OPTS=&quot;hostfwd=tcp::18080-:80,hostfwd=tcp::10022-:22&quot;
$ ./result/bin/run-vim-vm-vm
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Then, from a another terminal:&lt;/p&gt;

&lt;pre&gt;&lt;code class=&quot;bash&quot;&gt;$ ssh nixos@localhost -p 10022
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;And you should be in a freshly baked NixOS VM with your Vim config ready to be
used.&lt;/p&gt;

&lt;p&gt;There's an always current example of my &lt;a href=&quot;https://source.mcwhirter.io/craige/mio-ops/src/branch/master/roles/vim.nix&quot;&gt;production Vim
configuration&lt;/a&gt;
in my &lt;a href=&quot;https://source.mcwhirter.io/craige/mio-ops/&quot;&gt;mio-ops&lt;/a&gt; repo.&lt;/p&gt;</description>
	<pubDate>Thu, 14 Nov 2019 04:18:37 +0000</pubDate>
</item>

</channel>
</rss>