Optimize Nix release workflow#4927
Conversation
…shes into one job, building the shared derivations once The k-framework-binary (private, via kup) and k-framework (public, via cachix) publishes built the same three derivations on separate runners, recompiling K four times (2 jobs x 2 OS) for work that needs two builds. Fold them into one matrix job: the first publish populates the local Nix store so the second's nix build is an instant store hit. The two publishes stay independent (continue-on-error + if: always()) so a flaky upload to one cache neither blocks nor masks the other, and a final guard re-fails the job if either push failed, preserving the release gate the two separate jobs provided. Drop the removed cachix-release-dependencies from the release job's needs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
juliankuners
left a comment
juliankuners
left a comment
There was a problem hiding this comment.
See comments and suggestions.
| - name: 'Build and cache K and K dependencies' | ||
| - name: 'Build and cache K and K dependencies (public)' | ||
| id: dependencies | ||
| if: always() |
There was a problem hiding this comment.
if: always() is unnecessary, since the previous step already declared continue-on-error
| nix-store --query --requisites --include-outputs ${DRV_K_OPENSSL_PROCPS_SECP256K1} | cachix push k-framework | ||
| - name: 'Fail if either cachix publish failed' | ||
| if: always() |
There was a problem hiding this comment.
Same unnecessary if: always(), see above comment.
| # The binary publish and the dependency-closure publish build the same three | ||
| # derivations. Running them as sequential steps on one runner means the build | ||
| # happens once: the first step populates the local Nix store, so the second | ||
| # step's `nix build` is an instant store hit rather than a full recompile. | ||
| # | ||
| # The two publishes target different caches with different tokens (private | ||
| # `k-framework-binary` via kup vs. public `k-framework` via cachix), so they are | ||
| # kept independent: `continue-on-error` plus `if: always()` ensures a flaky | ||
| # upload to one cache neither blocks nor masks the other, and the final guard | ||
| # re-fails the job if either push failed — preserving the release gate that the | ||
| # two separate jobs used to provide. | ||
| - name: 'Publish K to k-framework-binary cache (private)' | ||
| id: binary | ||
| continue-on-error: true |
There was a problem hiding this comment.
Bloated AI comment that reasons why the old pattern of two workflows was merged into one. Once we've adopted this pattern everywhere, this comment will be confusing since it's not clear that the workflows was split in two in the first place.
If you wish to add a clarifying comment, keep it short and concise, simply just describing what the workflow does.
Though I agree that the differentian between the caches k-framework and k-framework-binary was never properly documented. Though I doubt that the proper place is a long comment in a workflow in one of our dependency projects. Rather, there should be proper documentation in possibly https://github.com/runtimeverification/rv-nix-tools/tree/master/docs.
| cachix-release: | ||
| name: 'k-framework-binary cachix release' | ||
| name: 'Cachix release (binary cache + dependency closure)' |
There was a problem hiding this comment.
| name: 'Cachix release (binary cache + dependency closure)' | |
| name: 'Cachix release (binary cache `k-framework-binary` + dependency cache `k-framework`)' |
Shorten the cachix-release comment to describe what the step does, drop the redundant if: always() on steps following a continue-on-error step, and name the caches explicitly in the job name. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3 participants
The k-framework-binary (private, via kup) and k-framework (public, via cachix) publishes built the same three derivations on separate runners, recompiling K four times (2 jobs x 2 OS) for work that needs two builds. Fold them into one matrix job: the first publish populates the local Nix store so the second's nix build is an instant store hit. The two publishes stay independent (continue-on-error + if: always()) so a flaky upload to one cache neither blocks nor masks the other, and a final guard re-fails the job if either push failed, preserving the release gate the two separate jobs provided. Drop the removed cachix-release-dependencies from the release job's needs.