Skip to content

Commit 88ed81f

Browse files
enobayramDevopsGoth
enobayram
and
DevopsGoth
authored
Implement actions for mac-m1 runners (#4)
* Restart the Nix daemon on MacOS * Use absolute Nix path in post-build-hook * Restore the env vars in post-build-hook * Perform upload as runner * Make upload-paths executable * tee upload-paths * Detach socat twice * Whitespace * Add debug lines * Wait more for nix shell * Fixing m1 mac (#3) We can drop the echo $PATH in the test job, that was just for diagnosis * Update action.yml * Update action.yml * Update action.yml * Update test.yml * Update test.yml * Update test.yml * Update test.yml * Update action.yml * Use ncat instead of socat * Add a sleep * Make a phony upload request * Add a sleep * Add shebang * Perform the actual upload * Let the build hook receive copy logs * Give root user access to aws credentials * Save credentials from environment * Read from env * Redirect to file in sudo * Let post-build-hook perform upload Now that we give root user access * Run nix with full path * Expose multiple actions for various use cases * Fix runner.name condition for install nix * Avoid ternary operator in env var expression * Allow substitution in root path * Set NIX_PATH for mac-m1 * Remove export from NIX_PATH string * Remove unnecessary conditions * Avoid the deprecated ::set-ouput * Add overriden nix.conf opts * Remove comment * Experiment * Try running a nix build * Try adding back post-build-hook * Pass nix_path option to install-nix-action * Experiment with adding sleep * Experiment with increasing sleep * Wait until nix daemon comes back * Update readme * Remove empty testfile * Add disclaimer for external users --------- Co-authored-by: June <38109440+DevopsGoth@users.noreply.github.com>
1 parent d977b8b commit 88ed81f

File tree

7 files changed

+258
-66
lines changed

7 files changed

+258
-66
lines changed

‎.github/workflows/test.yml

Lines changed: 36 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name: Test the GitHub action of this repository
33
on: [push]
44

55
jobs:
6-
test-build-and-cache:
6+
test-build-and-cache-by-root:
77
runs-on: ${{ matrix.os }}
88
strategy:
99
fail-fast: false
@@ -20,17 +20,14 @@ jobs:
2020
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
2121
aws-region: us-east-1
2222

23-
- name: Run this repo's action
24-
# For an external repo wishing to use this action, the uses field needs to be:
25-
# uses: kadena-io/setup-nix-with-cache@{The version you want to use}
26-
uses: ./
23+
- # For an external repo wishing to use this action, the uses field needs to be:
24+
# uses: kadena-io/setup-nix-with-cache/by-root@{The version you want to use}
25+
uses: ./by-root
2726
with:
2827
cache_url: s3://nodemon-nix-cache?region=us-east-1
2928
signing_private_key: ${{ secrets.NIX_CACHE_PRIVATE_KEY }}
3029

31-
# This step is probably not needed for external repos that use this action
32-
- name: Set up Nix path
33-
run: echo "NIX_PATH=nixpkgs=https://github.com/NixOS/nixpkgs/archive/22.11.tar.gz" >> $GITHUB_ENV
30+
- uses: ./copy-root-aws-credentials
3431

3532
- name: Attempt to build a new derivation with Nix
3633
run: |
@@ -40,3 +37,34 @@ jobs:
4037
run: |
4138
nix-build -E '(import <nixpkgs> {}).runCommand "fetch-example" {} "echo Example > $out"'
4239
40+
test-build-and-cache-by-runner:
41+
runs-on: ${{ matrix.os }}
42+
strategy:
43+
fail-fast: false
44+
matrix:
45+
os: [ubuntu-latest, macos-latest, mac-m1]
46+
steps:
47+
- name: Checkout repository
48+
uses: actions/checkout@v2
49+
50+
- name: Set up AWS credentials
51+
uses: aws-actions/configure-aws-credentials@v1
52+
with:
53+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
54+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
55+
aws-region: us-east-1
56+
57+
- uses: ./by-runner
58+
with:
59+
cache_url: s3://nodemon-nix-cache?region=us-east-1
60+
signing_private_key: ${{ secrets.NIX_CACHE_PRIVATE_KEY }}
61+
62+
- uses: ./copy-root-aws-credentials
63+
64+
- name: Attempt to build a new derivation with Nix
65+
run: |
66+
nix-build -E '(import <nixpkgs> {}).writeText "build-example" (builtins.toString builtins.currentTime)'
67+
68+
- name: Attempt to fetch an existing cache entry
69+
run: |
70+
nix-build -E '(import <nixpkgs> {}).runCommand "fetch-example" {} "echo Example > $out"'

‎README.md

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,19 @@
1-
# Set up Nix along with a custom cache
1+
*NOTE*: The actions in this repository are meant to be used with Kadena's own github actions infrastructure, therefore they contain Kadena-specific logic. They may still be useful for external users out of the box or after some tweaks, but that's not officially supported by Kadena.
22

3-
GitHub action for setting up a simple Nix environment that caches to a custom Nix cache
3+
# Set up Nix along with caching
44

5-
## Usage
5+
A set of GitHub actions for setting up a multi-user Nix environment with a binary cache for uploading and fetching derivation outputs.
6+
7+
The repository contains the following actions:
8+
* `by-root`: Sets up `nix.conf` with a `post-build-hook` so that intermediate and final build outputs are uploaded by the root.
9+
* `by-runner`: Starts a background upload job as the action runner and sets up `nix.conf` with a `post-build-hook` so that intermediate and final build outputs are passed to the action runner for cache uploads.
10+
* `copy-root-aws-credentials`: Copies the AWS credentials of the action runner to the root user. This is meant to be used for giving the root user access to binary caches. In the case of a `by-root` workflow, this can be used to give root write permissions and in the case of a private cache, it can be used to give root read permissions.
11+
12+
Depending on the binary cache in question, these actions can be used together in a number of ways, some example use cases:
13+
* A public AWS S3 bucket: A `by-root` + `copy-root-aws-credentials` setup is recommended. One can also use `by-runner` only, but `by-runner` adds around a minute of setup time to each run.
14+
* A private AWS S3 bucket: `by-root` + `copy-root-aws-credentials` will work same as before. If `by-runner` is preferred, it will also `copy-root-aws-credentials` since otherwise the Nix builder won't be able to read from the private S3 bucket.
15+
* Other kind of cache (e.g. over SSH, etc.): As long as the github action runner has access to the cache, `by-runner` can be used to enable cache uploads, but if the cache is private, a custom step will be required to give the root user access to it. If root is given access, then `by-root` can also be used instead of `by-runner` in order to avoid the setup overhead.
16+
17+
## Example
618

719
Please check [.github/workflows/test.yml](.github/workflows/test.yml) for an example using an AWS S3 bucket as a Nix cache.

‎action.yml

Lines changed: 0 additions & 55 deletions
This file was deleted.

‎by-root/action.yml

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
name: 'Set up Nix environment with cache uploads by the root'
2+
description: -|
3+
This action installs Nix in multi-user mode and configures it to
4+
use a custom cache. It also uploads all the build results to the cache,
5+
including intermediate packages built as part of a build activity.
6+
The cache uploads are performed by the root user in the post-build-hook,
7+
this means that the root user needs to have the proper credentials
8+
configured to have write access to the cache.
9+
inputs:
10+
cache_url:
11+
description: 'URL for the Nix cache'
12+
required: true
13+
signing_private_key:
14+
description: The private (secret) key used for signing Nix store paths
15+
required: true
16+
runs:
17+
using: composite
18+
steps:
19+
- name: Install Nix
20+
uses: cachix/install-nix-action@v22
21+
with:
22+
nix_path: nixpkgs=https://github.com/NixOS/nixpkgs/archive/23.05.tar.gz
23+
24+
- name: Set NIX_PATH
25+
run: |
26+
echo "NIX_PATH=nixpkgs=https://github.com/NixOS/nixpkgs/archive/23.05.tar.gz" >> $GITHUB_ENV
27+
shell: bash
28+
29+
- run: |
30+
echo $PATH
31+
nix-build --version
32+
shell: bash
33+
34+
- name: Populate the nix.conf with cache fields
35+
shell: bash
36+
run: |
37+
TMP_DIR=$(mktemp -d)
38+
echo "${{ inputs.signing_private_key }}" > "$TMP_DIR/key.private"
39+
40+
tee $TMP_DIR/post-build-hook.sh <<EOF
41+
#!/usr/bin/env bash
42+
43+
set -euo pipefail
44+
set -f # disable globbing
45+
echo "Built paths:" \$OUT_PATHS
46+
$(which nix) copy --to "${{ inputs.cache_url }}" \$OUT_PATHS 2>&1
47+
EOF
48+
chmod a+x $TMP_DIR/post-build-hook.sh
49+
50+
CERTFILEOPT=$( [[ "$OSTYPE" =~ darwin ]] && echo "ssl-cert-file = /etc/ssl/cert.pem" || echo "" )
51+
52+
sudo mkdir -p /etc/nix/
53+
sudo tee /etc/nix/nix.conf <<EOF
54+
show-trace = true
55+
max-jobs = auto
56+
$CERTFILEOPT
57+
trusted-users = root ${USER:-}
58+
substituters = ${{ inputs.cache_url }} https://cache.nixos.org/
59+
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
60+
secret-key-files = $TMP_DIR/key.private
61+
post-build-hook = $TMP_DIR/post-build-hook.sh
62+
experimental-features = nix-command fetch-closure flakes
63+
EOF
64+
65+
- name: Restart the Nix daemon on MacOS
66+
shell: bash
67+
if: ${{ runner.os == 'macOS' }}
68+
run: |
69+
sudo launchctl stop org.nixos.nix-daemon
70+
sudo launchctl start org.nixos.nix-daemon
71+
while ! nix store ping 2>/dev/null; do
72+
sleep 1
73+
done
74+
75+
- name: Restart the Nix daemon on Linux
76+
shell: bash
77+
if: ${{ runner.os == 'Linux' }}
78+
run: |
79+
sudo systemctl restart nix-daemon.service

‎by-runner/action.yml

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
name: 'Set up Nix environment with cache uploads by the runner'
2+
description: -|
3+
This action installs Nix in multi-user mode and configures it to
4+
use a custom cache. It also uploads all the build results to the cache,
5+
including intermediate packages built as part of a build activity.
6+
The cache uploads are performed by the runner user in the post-build-hook,
7+
this means that the runner user needs to have the proper credentials
8+
configured to have write access to the cache.
9+
inputs:
10+
cache_url:
11+
description: 'URL for the Nix cache'
12+
required: true
13+
signing_private_key:
14+
description: The private (secret) key used for signing Nix store paths
15+
required: true
16+
runs:
17+
using: composite
18+
steps:
19+
- name: Install Nix
20+
uses: cachix/install-nix-action@v22
21+
with:
22+
nix_path: nixpkgs=https://github.com/NixOS/nixpkgs/archive/23.05.tar.gz
23+
24+
- name: Set NIX_PATH
25+
run: |
26+
echo "NIX_PATH=nixpkgs=https://github.com/NixOS/nixpkgs/archive/23.05.tar.gz" >> $GITHUB_ENV
27+
shell: bash
28+
29+
- run: |
30+
echo $PATH
31+
nix-build --version
32+
shell: bash
33+
34+
- name: Populate the nix.conf with cache fields
35+
shell: bash
36+
run: |
37+
TMP_DIR=$(mktemp -d)
38+
echo "${{ inputs.signing_private_key }}" > "$TMP_DIR/key.private"
39+
40+
tee $TMP_DIR/upload-paths.sh <<EOF
41+
#!/usr/bin/env bash
42+
read -r OUT_PATHS
43+
export IFS=' '
44+
echo Uploading paths \$OUT_PATHS
45+
nix copy --to "${{ inputs.cache_url }}" \$OUT_PATHS 2>&1
46+
EOF
47+
chmod a+x $TMP_DIR/upload-paths.sh
48+
49+
NMAP=$(nix-build '<nixpkgs>' -A nmap --no-out-link)
50+
$NMAP/bin/ncat -k -l 54321 -e $TMP_DIR/upload-paths.sh &
51+
52+
sleep 1
53+
echo $NMAP | nc localhost 54321
54+
55+
tee $TMP_DIR/post-build-hook.sh <<EOF
56+
#!/usr/bin/env bash
57+
58+
set -euo pipefail
59+
set -f # disable globbing
60+
echo "Built paths:" \$OUT_PATHS
61+
echo \$OUT_PATHS | nc localhost 54321
62+
EOF
63+
chmod a+x $TMP_DIR/post-build-hook.sh
64+
65+
CERTFILEOPT=$( [[ "$OSTYPE" =~ darwin ]] && echo "ssl-cert-file = /etc/ssl/cert.pem" || echo "" )
66+
67+
sudo mkdir -p /etc/nix/
68+
sudo tee /etc/nix/nix.conf <<EOF
69+
show-trace = true
70+
max-jobs = auto
71+
$CERTFILEOPT
72+
trusted-users = root ${USER:-}
73+
substituters = ${{ inputs.cache_url }} https://cache.nixos.org/
74+
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
75+
secret-key-files = $TMP_DIR/key.private
76+
post-build-hook = $TMP_DIR/post-build-hook.sh
77+
experimental-features = nix-command fetch-closure flakes
78+
EOF
79+
80+
- name: Restart the Nix daemon on MacOS
81+
shell: bash
82+
if: ${{ runner.os == 'macOS' }}
83+
run: |
84+
sudo launchctl stop org.nixos.nix-daemon
85+
sudo launchctl start org.nixos.nix-daemon
86+
while ! nix store ping 2>/dev/null; do
87+
sleep 1
88+
done
89+
90+
- name: Restart the Nix daemon on Linux
91+
shell: bash
92+
if: ${{ runner.os == 'Linux' }}
93+
run: |
94+
sudo systemctl restart nix-daemon.service

‎copy-root-aws-credentials/action.yml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: 'Copy the AWS credentials to the root user'
2+
description: -|
3+
This action copies the AWS credentials from the runner user to
4+
the root user. This is meant to be used for actions where the
5+
Nix daemon needs to read from or write to a binary cache that
6+
requires authentication.
7+
runs:
8+
using: composite
9+
steps:
10+
- name: Determine AWS root path
11+
id: aws-root-path
12+
shell: bash
13+
run: |
14+
if [[ "${{ runner.os }}" == "macOS" ]]; then
15+
echo "path=/var/root/.aws" >> $GITHUB_OUTPUT
16+
else
17+
echo "path=/root/.aws" >> $GITHUB_OUTPUT
18+
fi
19+
20+
- name: Copy aws credentials to root and clean up at the end
21+
uses: pyTooling/Actions/with-post-step@v0.4.5
22+
env:
23+
AWS_ROOT_PATH: ${{ steps.aws-root-path.outputs.path }}
24+
with:
25+
main: |
26+
echo Copy aws credentials to $AWS_ROOT_PATH
27+
sudo mkdir -p $AWS_ROOT_PATH
28+
sudo sh -c "cat - > $AWS_ROOT_PATH/credentials" <<EOF
29+
[default]
30+
aws_access_key_id = $AWS_ACCESS_KEY_ID
31+
aws_secret_access_key = $AWS_SECRET_ACCESS_KEY
32+
EOF
33+
post: |
34+
sudo rm -rf $AWS_ROOT_PATH

‎testfile

Whitespace-only changes.

0 commit comments

Comments
 (0)