v0.2.2

What's Changed

• client: share one retry policy across network calls by @Mic92 in #217
• client: retry FetchNars on connect failures and broken streams by @Mic92 in #218
• Release 0.2.2 by @Mic92 in #220

Full Changelog: v0.2.1...v0.2.2

v0.2.1

Add workers from a Kubernetes cluster to an existing farm

The chart can now run only workers that join a scheduler outside the cluster, for example a farm deployed with the NixOS modules. The release then has no scheduler and no balancer. Each worker takes a port on its node, since the external balancer usually cannot reach pod IPs:

scheduler:
  external: farm.example.com:50051
workers:
  x86-64:
    system: x86_64-linux
    hostPort: 50052
    metricsHostPort: 9465 # optional, for a scraper outside the cluster
    advertise: # optional: node name -> IP the balancer reaches it at
      node-1: 10.0.0.1

Without advertise, workers advertise their node's InternalIP. See docs/kubernetes.md, which now also covers running KVM builds (nixos-test) through a device plugin.

Daemon confined with Landlock and Seatbelt

--sandbox restricts the daemon right after startup: it may write only $TMPDIR, execute only from the store, connect to the nix-daemon socket and listen on its own addresses. It is on by default in the NixOS and launchd modules.

Faster builds through the client

• Derivations are uploaded once before scheduling, instead of each build in a chain waiting for its own .drv upload.
• Valid paths are queried top-down from the roots, so a store that already holds a closure answers in a few small calls instead of one slow one for the whole closure.

Helm

• sandbox.runtimeClassName selects a RuntimeClass, for nodes whose containerd gives pods a writable cgroup (per-build memory limits).
• A worker reseeds its store when the image changed, so a moving tag no longer leaves /bin pointing at missing store paths.

Fixes

• nix-daemon restarts: a worker recovers instead of answering "previously failed" until it is restarted, and its readiness checks the local nix-daemon, so a worker with a dead daemon leaves the balancer.
• macOS: the daemon's file descriptor limit is 65536 instead of launchd's 256, which stalled TLS handshakes on busy workers.
• Builds against Nix 2.36pre: the client compiles against versions with substituter.hh but without the batching Store::addTempRoots.

The farm dashboard shows fleet totals and per-system lines instead of one line per worker, so it stays readable with many builders.

Verify the images:

gh attestation verify oci://ghcr.io/mic92/nix-grpc-farm:v0.2.1 -R Mic92/nix-grpc-store

Contributors

Thanks to @GaetanLepage for the nix-daemon recovery fix and to @zimbatm for the Nix 2.36pre build fix.

Full Changelog: v0.2.0...v0.2.1

v0.2.0

Signed container images

The worker, balancer and client images carry GitHub build provenance:

gh attestation verify oci://ghcr.io/mic92/nix-grpc-farm:v0.2.0 -R Mic92/nix-grpc-store

Use nix-grpc-farm-lb and nix-grpc-farm-client for the other two.

Clients verify a normal certificate

The balancer used one certificate for clients and for the nodes, so clients outside the cluster needed the farm CA. Give clients a public certificate, for example from Let's Encrypt, and keep the farm CA for the nodes:

tls:
  certManager:
    enabled: true
    public:
      issuerRef: {name: letsencrypt, kind: ClusterIssuer}
      dnsNames: [farm.example.com]

Or point tls.public.existingSecret at your own secret. Nothing changes without these options.

Gateway API in front of the balancer

Cilium only talks HTTP/2 to a backend whose port says so:

lb:
  service:
    appProtocol: kubernetes.io/h2c

One pod per node

spread: required makes worker groups and the balancer insist on it. Taints are honored when counting nodes, so a control plane node no longer blocks rolling updates:

workerDefaults:
  spread: required

Nix tools with the plugin for your nixpkgs

nix, nix-eval-jobs and nix-fast-build built from your pkgs, for a dev shell:

let
  tools = nix-grpc-store.lib.nixToolsWithPlugin pkgs;
in
pkgs.mkShell { packages = [ tools.nix tools.nix-eval-jobs tools.nix-fast-build ]; }

Fixes

• Ctrl-C: every client call cancels on SIGINT. During AddMultipleToStore an interrupt used to be ignored for minutes.
• Failed NAR fetches: the next request reaches the server again. Before, a failure stayed cached, so nix repl after ^C could never fetch that path.
• Flux: charts from an OCIRepository (<version>+<digest>) apply again, because the helm.sh/chart label no longer contains +.

Contributors

Thanks to @Kranzes for the Helm chart options (public certificate, appProtocol, required spread, Flux label) and the Ctrl-C fix, and to @GaetanLepage for nixToolsWithPlugin. @Kranzes made their first contribution in #167.

Full Changelog: v0.1.0...v0.2.0

v0.1.0

What's Changed

• This is the first stable release of nix-grpc-store. See https://blog.thalheim.io/2026/09/23/scaling-nix-builds-across-a-fleet-with-nix-grpc-store-and-niks3/ for the announcement blog post.

New Contributors

• @GaetanLepage made their first contribution in #157

Full Changelog: v0.1.0-beta.4...v0.1.0

v0.1.0-beta.4

What's Changed

• lb: ping downstream every 10s by @zimbatm in #118
• scheduler: send the per-system gauges that the throttle skipped by @zimbatm in #119
• dispatcher: publish which builders are connected by @Mic92 in #126
• [need #126] auth: identify a peer by its own certificate by @Mic92 in #127
• daemon: serve envoy the list of builders by @Mic92 in #128
• lb: take the builders from the scheduler by @Mic92 in #129
• [need #129] lb: keep the builder list from clients by @Mic92 in #130
• [need #130] docs: explain how the balancer finds builders by @Mic92 in #131
• build with clang, use PCH only there by @Mic92 in #134
• nixos: add a nix-darwin module by @Mic92 in #135
• docs: draw the diagrams without an SVG filter by @Mic92 in #136
• flake.lock: niks3 1.13.0 by @Mic92 in #137
• Release 0.1.0-beta.4 by @Mic92 in #138

Full Changelog: v0.1.0-beta.3...v0.1.0-beta.4

v0.1.0-beta.3

Upgrade notes

• nix store gc and nix store delete against a grpc:// store now fail with an error instead of running on whichever worker the balancer picked. Workers collect their own stores. (#114)
• The bundled niks3 is now 1.12.0 (#116).

Fixes

• A worker that vanished without closing its connection (poweroff, crash, network loss) stayed registered with the scheduler for about 15 minutes, and clients kept retrying builds on it. The balancer now pings downstream connections every 30s and closes them after 10s without an answer. (#115, thanks @zimbatm)
• A wrong ca-cert on the client now blames the server certificate. The old hint claimed no client certificate was presented. (#112)
• A typo in requiredSystemFeatures still waits out the restart grace, since workers may be starting, but the message now lists the features the workers offer. (#113)

Full Changelog: v0.1.0-beta.2...v0.1.0-beta.3

v0.1.0-beta.2

Upgrade notes

• Worker pods no longer run a privileged container. They need Kubernetes 1.33 or newer (user namespaces). On older clusters set sandbox.privileged=true. See Sandbox and pod privileges.
• Signatures on fresh outputs need niks3 1.12.0-beta.3 or newer. With an older niks3 everything else works, the outputs just carry no cache signature.

New

• nix-with-plugin and lib.wrapNix put the grpc:// store scheme into a Nix of your choice (#98)
• TLS certificates renewed in place (cert-manager) are picked up without a restart, in the daemon and the balancer (#95, #96)

Fixes

• --timeout and --max-silent-time now reach the builder (#107)
• A build keeps running while any client still waits for it, so the first client leaving no longer fails the second (#106)
• Outputs read back through any worker carry the cache's signature, so signature-checking clients accept them (#108)
• Clients survive a balancer restart mid-build (#100)
• Workers recover from a local nix-daemon restart (#101)
• Shorter gap after a scheduler handover (#102), and the scheduler keeps leading across a niks3 restart (#103)

Full Changelog: v0.1.0-beta.1...v0.1.0-beta.2

v0.1.0-beta.1

What's Changed

• docs: link a screenshot of the farm dashboard by @Mic92 in #83
• daemon: read the CA bundle for the scheduler channel instead of passing its path by @Mic92 in #81
• dashboard: busy slots per system, builds per hour by @Mic92 in #84
• helm: cert-manager PKI, spread pods over nodes, VMPodScrape by @Mic92 in #85
• helm: fix worker drain on pod delete by @Mic92 in #87
• client image for CI jobs in the cluster by @Mic92 in #88
• docs: separate Kubernetes guide by @Mic92 in #89
• [need #89] client image: drop niks3, nix-fast-build 2.0.3, build with --store by @Mic92 in #91
• release: allow pre-releases, clean up client image tags by @Mic92 in #92
• Release 0.1.0-beta.1 by @Mic92 in #93

Full Changelog: assets...v0.1.0-beta.1

assets

Images linked from the documentation. Not a software release.