add guide on post build hook (#528)

* add guide on post build hook

originally written under LGPL 2.1 for the Nix reference manual.
authors as recorded in the Nix repository:

    git log --pretty="Co-authored-by: %an <%ae>" -- doc/manual/advanced-topics/post-build-hook.xml doc/manual/src/advanced-topics/post-build-hook.md | sort | uniq

relicenced with authors' permission to CC-BY-SA 4.0: #463

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Graham Christensen <graham@grahamc.com>
Co-authored-by: Ellie Hermaszewska <git@monoid.al>
Co-authored-by: Kevin Stock <kevin@kevinstock.org>
Co-authored-by: endgame <endgame@users.noreply.github.com>
Co-authored-by: regnat <rg@regnat.ovh>

* WIP: reword, reformat, fix links

* rebase 528 onto current master, move recipe to correct directory

* fix header case to make vale happy

* fix links after moving from nix repo

* slight rewordings for clarity

Co-authored-by: wamirez <wamirez@protonmail.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

---------

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Graham Christensen <graham@grahamc.com>
Co-authored-by: Ellie Hermaszewska <git@monoid.al>
Co-authored-by: Kevin Stock <kevin@kevinstock.org>
Co-authored-by: endgame <endgame@users.noreply.github.com>
Co-authored-by: regnat <rg@regnat.ovh>
Co-authored-by: alex <source@proof.construction>
Co-authored-by: wamirez <wamirez@protonmail.com>

source/guides/recipes/index.md

@@ -9,4 +9,5 @@ Automatic environments <direnv>
 sharing-dependencies.md
 Managing remote sources <./dependency-management.md>
 Python development environment <./python-environment.md>
+post-build-hook.md
 ```

source/guides/recipes/post-build-hook.md

@@ -0,0 +1,118 @@
+# Upload build results to S3
+
+This guide shows how to use the Nix [`post-build-hook`](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-post-build-hook) configuration option to automatically upload build results to an S3-compatible binary cache.
+
+## Implementation caveats
+
+This is a simple and working example, but it is not suitable for all use cases.
+
+The post-build hook program runs after each executed build, and blocks the build loop.
+The build loop exits if the hook program fails.
+
+Concretely, this implementation will make Nix slow or unusable when the network connection is slow or unreliable.
+A more advanced implementation might pass the store paths to a user-supplied daemon or queue for processing the store paths outside of the build loop.
+
+# Prerequisites
+
+<!-- TODO: this information will move: https://github.com/NixOS/nix/issues/7769 -->
+This tutorial assumes you have [configured an S3-compatible binary cache](https://nixos.org/manual/nix/stable/package-management/s3-substituter.html), and that the `root` user's default AWS profile can upload to the bucket.
+
+# Set up a signing key
+
+Use [`nix-store --generate-binary-cache-key`](https://nixos.org/manual/nix/stable/command-ref/nix-store/generate-binary-cache-key.html) to create a pair of cryptographic keys.
+You will sign paths with the private key, and distribute the public key for verifying the authenticity of the paths.
+
+```console
+$ nix-store --generate-binary-cache-key example-nix-cache-1 /etc/nix/key.private /etc/nix/key.public
+$ cat /etc/nix/key.public
+example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM=
+```
+
+Then update [`nix.conf`](https://nixos.org/manual/nix/stable/command-ref/conf-file.html) on any machine that will access the cache.
+Add the cache URL to [`substituters`](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-substituters) and the public key to [`trusted-public-keys`](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-trusted-public-keys):
+
+    substituters = https://cache.nixos.org/ s3://example-nix-cache
+    trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM=
+
+Machines that build for the cache must sign derivations using the private key.
+The path to the file containing the private key you just generated must be added to the [`secret-key-files`](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-secret-key-files) field in the [`nix.conf`](https://nixos.org/manual/nix/stable/command-ref/conf-file.html) of those machines:
+
+    secret-key-files = /etc/nix/key.private
+
+You will restart the Nix daemon in a later step.
+
+# Implementing the build hook
+
+Write the following script to `/etc/nix/upload-to-cache.sh`:
+
+```bash
+#!/bin/sh
+set -eu
+set -f # disable globbing
+export IFS=' '
+echo "Uploading paths" $OUT_PATHS
+exec nix copy --to "s3://example-nix-cache" $OUT_PATHS
+```
+
+> **Note**
+>
+> The `$OUT_PATHS` variable is a space-separated list of Nix store
+> paths. In this case, we expect and want the shell to perform word
+> splitting to make each output path its own argument to `nix
+> store sign`. Nix guarantees the paths will not contain any spaces,
+> however a store path might contain glob characters. The `set -f`
+> disables globbing in the shell.
+Then make sure the hook program is executable by the `root` user:
+
+```console
+# chmod +x /etc/nix/upload-to-cache.sh
+```
+
+# Updating nix configuration
+
+Edit `/etc/nix/nix.conf` to run our hook, by adding the following
+configuration snippet at the end:
+
+    post-build-hook = /etc/nix/upload-to-cache.sh
+
+Then, restart the `nix-daemon`.
+
+# Testing
+
+Build any derivation, for example:
+
+```console
+$ nix-build -E '(import <nixpkgs> {}).writeText "example" (builtins.toString builtins.currentTime)'
+this derivation will be built:
+  /nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv
+building '/nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv'...
+running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
+post-build-hook: Signing paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+post-build-hook: Uploading paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+/nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+```
+
+Then delete the path from the store, and try substituting it from the
+binary cache:
+
+```console
+$ rm ./result
+$ nix-store --delete /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+```
+
+Now, copy the path back from the cache:
+
+```console
+$ nix-store --realise /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
+copying path '/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example from 's3://example-nix-cache'...
+warning: you did not specify '--add-root'; the result might be removed by the garbage collector
+/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example
+```
+
+# Conclusion
+
+We now have a Nix installation configured to automatically sign and
+upload every local build to a remote binary cache.
+
+Before deploying this to production, be sure to consider the
+[implementation caveats](#implementation-caveats).