Add workflow for building runner image

.github/workflows/e2e_test.yaml

@@ -492,3 +492,5 @@ jobs:
         run: |
           HOST_IP=$(ip route | grep default | cut -f 3 -d" ")
           [ $((ping $HOST_IP -c 5 || :) | grep "Destination Port Unreachable" | wc -l) -eq 5 ]
+      - name: Test sctp support
+        run: sudo apt-get install lksctp-tools -yq && checksctp

charmcraft.yaml

@@ -11,6 +11,8 @@ parts:
       - libffi-dev  # for cffi
       - libssl-dev  # for cryptography
       - rust-all  # for cryptography
+    prime:
+      - scripts/build-image.sh
 bases:
   - build-on:
     - name: "ubuntu"

docs/explanation/charm-architecture.md

@@ -5,6 +5,7 @@ A [Juju](https://juju.is/) [charm](https://juju.is/docs/olm/charmed-operators) t
 Conceptually, the charm can be divided into the following:
 
 - Management of LXD ephemeral virtual machines to host [ephemeral self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/autoscaling-with-self-hosted-runners#using-ephemeral-runners-for-autoscaling)
+- Management of the virtual machine image
 - Management of the network
 - GitHub API usage
 - Management of [Python web service for checking GitHub repository settings](https://github.com/canonical/repo-policy-compliance)
@@ -20,6 +21,31 @@ On schedule or upon configuration change, the charm performs a reconcile to ensu
 
 The virtual machines hosting the runner use random access memory as disk; therefore, the [`vm-disk` configuration](https://charmhub.io/github-runner/configure#vm-disk) can impact the memory usage of the Juju machine. This is done to prevent disk IO exhaustion on the Juju machine on disk-intensive GitHub workflows. In the future, an alternative method to prevent disk IO exhaustion will be implemented.
 
+## Virtual machine image
+
+The virtual machine images are built on installation and on a schedule every 6 hours. These images are constructed by launching a virtual machine instance, modifying the instance with configurations and software installs, and then exporting the instance as an image. This process reduces the time needed to launch a virtual machine instance for hosting the self-hosted runner application.
+
+The software installed in the image includes:
+
+- APT packages:
+  - docker.io
+  - npm
+  - python3-pip
+  - shellcheck
+  - jq
+  - wget
+- npm packages:
+  - yarn
+- Binary downloaded:
+  - yq
+
+The configurations applied in the image include:
+
+- Creating a group named `microk8s`.
+- Adding the `ubuntu` user to the `microk8s` group. Note that the `microk8s` package is not installed in the image; this preconfigures the group for users who install the package.
+- Adding the `ubuntu` user to the `docker` group.
+- Adding iptables rules to accept traffic for the DOCKER-USER chain. This resolves a networking conflict with LXD.
+
 ## Network configuration
 
 The charm respects the HTTP(S) proxy configuration of the model configuration of Juju. The configuration can be set with [`juju model-config`](https://juju.is/docs/juju/juju-model-config) using the following keys: `juju-http-proxy`, `juju-https-proxy`, `juju-no-proxy`. The GitHub self-hosted runner applications are configured to use the proxy configuration.
@@ -28,7 +54,6 @@ If an HTTP(S) proxy is used, all HTTP(S) requests in the GitHub workflow will be
 
 The nftables on the Juju machine are configured to deny traffic from the runner virtual machine to IPs on the [`denylist` configuration](https://charmhub.io/github-runner/configure#denylist). The runner will always have access to essential services such as DHCP and DNS, regardless of the denylist configuration.
 
-
 ## GitHub API usage
 
 The charm requires a GitHub personal access token for the [`token` configuration](https://charmhub.io/github-runner/configure#token). This token is used for:

scripts/build-image.sh

@@ -0,0 +1,104 @@
+#!/usr/bin/env bash
+
+# Copyright 2023 Canonical Ltd.
+# See LICENSE file for licensing details.
+
+set -e
+
+retry() {
+    local command="$1"
+    local wait_message="$2"
+    local max_try="$3"
+
+    local attempt=0
+
+    while ! $command
+    do
+        attempt=$((attempt + 1))
+        if [[ attempt -ge $max_try ]]; then
+            return
+        fi
+
+        echo "$wait_message"
+        sleep 10
+    done
+}
+
+cleanup() {
+    local test_command="$1"
+    local clean_up_command="$2"
+    local wait_message="$3"
+    local max_try="$4"
+
+    local attempt=0
+
+    while bash -c "$test_command"
+    do
+        echo "$wait_message"
+
+        $clean_up_command
+
+        attempt=$((attempt + 1))
+        if [[ attempt -ge $max_try ]]; then
+            # Cleanup failure.
+            return 1
+        fi
+
+        sleep 10
+    done
+}
+
+cleanup '/snap/bin/lxc info builder &> /dev/null' '/snap/bin/lxc delete builder --force' 'Cleanup LXD VM of previous run' 10
+
+if [[ "$1" == "test" ]]; then
+    retry '/snap/bin/lxc launch ubuntu-daily:jammy builder --device root,size=5GiB' 'Starting LXD VM'
+else
+    retry '/snap/bin/lxc launch ubuntu-daily:jammy builder --vm --device root,size=5GiB' 'Starting LXD container'
+fi
+retry '/snap/bin/lxc exec builder -- /usr/bin/who' 'Wait for lxd agent to be ready' 30
+retry '/snap/bin/lxc exec builder -- /usr/bin/nslookup github.com' 'Wait for network to be ready' 30
+
+/snap/bin/lxc exec builder -- /usr/bin/apt-get update
+/snap/bin/lxc exec builder --env DEBIAN_FRONTEND=noninteractive -- /usr/bin/apt-get upgrade -yq
+/snap/bin/lxc exec builder --env DEBIAN_FRONTEND=noninteractive -- /usr/bin/apt-get install linux-generic-hwe-22.04 -yq
+
+/snap/bin/lxc restart builder
+retry '/snap/bin/lxc exec builder -- /usr/bin/who' 'Wait for lxd agent to be ready' 30
+retry '/snap/bin/lxc exec builder -- /usr/bin/nslookup github.com' 'Wait for network to be ready' 30
+
+/snap/bin/lxc exec builder -- /usr/bin/apt-get update
+/snap/bin/lxc exec builder --env DEBIAN_FRONTEND=noninteractive -- /usr/bin/apt-get upgrade -yq
+/snap/bin/lxc exec builder --env DEBIAN_FRONTEND=noninteractive -- /usr/bin/apt-get install docker.io npm python3-pip shellcheck jq wget -yq
+/snap/bin/lxc exec builder -- /usr/bin/npm install --global yarn 
+/snap/bin/lxc exec builder -- /usr/sbin/groupadd microk8s
+/snap/bin/lxc exec builder -- /usr/sbin/usermod -aG microk8s ubuntu
+/snap/bin/lxc exec builder -- /usr/sbin/usermod -aG docker ubuntu
+/snap/bin/lxc exec builder -- /usr/sbin/iptables -I DOCKER-USER -j ACCEPT
+
+# Download and verify checksum of yq
+if [[ $(uname -m) == 'aarch64' ]]; then
+    YQ_ARCH="arm64"
+elif [[ $(uname -m) == 'arm64' ]]; then
+    YQ_ARCH="arm64"
+elif [[ $(uname -m) == 'x86_64' ]]; then
+    YQ_ARCH="amd64"
+else
+    echo "Unsupported CPU architecture: $(uname -m)"
+    return 1
+fi
+/usr/bin/wget "https://github.com/mikefarah/yq/releases/latest/download/yq_linux_$YQ_ARCH"
+/usr/bin/wget https://github.com/mikefarah/yq/releases/latest/download/checksums
+/usr/bin/wget https://github.com/mikefarah/yq/releases/latest/download/checksums_hashes_order
+/usr/bin/wget https://github.com/mikefarah/yq/releases/latest/download/extract-checksum.sh
+/usr/bin/bash extract-checksum.sh SHA-256 "yq_linux_$YQ_ARCH" | /usr/bin/awk '{print $2,$1}' | /usr/bin/sha256sum -c | /usr/bin/grep OK
+/snap/bin/lxc file push "yq_linux_$YQ_ARCH" builder/usr/bin/yq --mode 755
+
+/snap/bin/lxc publish builder --alias builder --reuse -f
+
+# Swap in the built image
+/snap/bin/lxc image alias rename runner old-runner || true
+/snap/bin/lxc image alias rename builder runner
+/snap/bin/lxc image delete old-runner || true
+
+# Clean up LXD instance
+cleanup '/snap/bin/lxc info builder &> /dev/null' '/snap/bin/lxc delete builder --force' 'Cleanup LXD VM' 10

src-docs/charm_state.py.md

@@ -32,7 +32,7 @@ Raised when charm config is invalid.
 
  - <b>`msg`</b>:  Explanation of the error. 
 
-<a href="../src/charm_state.py#L40"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="../src/charm_state.py#L41"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ### <kbd>function</kbd> `__init__`
 
@@ -77,7 +77,7 @@ Return the aproxy address.
 
 ---
 
-<a href="../src/charm_state.py#L98"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="../src/charm_state.py#L99"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ### <kbd>classmethod</kbd> `check_fields`
 
@@ -89,7 +89,7 @@ Validate the proxy configuration.
 
 ---
 
-<a href="../src/charm_state.py#L64"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="../src/charm_state.py#L65"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ### <kbd>classmethod</kbd> `from_charm`
 
@@ -122,14 +122,14 @@ The charm state.
 
  - <b>`is_metrics_logging_available`</b>:  Whether the charm is able to issue metrics. 
  - <b>`proxy_config`</b>:  Whether aproxy should be used. 
- - <b>`arch`</b>:  The underlying compute architecture, i.e. x64, amd64 
+ - <b>`arch`</b>:  The underlying compute architecture, i.e. x86_64, amd64, arm64/aarch64. 
 
 
 
 
 ---
 
-<a href="../src/charm_state.py#L159"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="../src/charm_state.py#L160"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ### <kbd>classmethod</kbd> `from_charm`
 
@@ -162,7 +162,7 @@ Raised when given machine charm architecture is unsupported.
 
  - <b>`arch`</b>:  The current machine architecture. 
 
-<a href="../src/charm_state.py#L117"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="../src/charm_state.py#L118"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ### <kbd>function</kbd> `__init__`
 

src-docs/github_client.py.md

@@ -0,0 +1,144 @@
+<!-- markdownlint-disable -->
+
+<a href="../src/github_client.py#L0"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+# <kbd>module</kbd> `github_client.py`
+GitHub API client. 
+
+Migrate to PyGithub in the future. PyGithub is still lacking some API such as remove token for runner. 
+
+
+
+---
+
+## <kbd>class</kbd> `GithubClient`
+GitHub API client. 
+
+<a href="../src/github_client.py#L25"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+### <kbd>function</kbd> `__init__`
+
+```python
+__init__(token: str, request_session: Session)
+```
+
+Instantiate the GiHub API client. 
+
+
+
+**Args:**
+
+ - <b>`token`</b>:  GitHub personal token for API requests. 
+ - <b>`request_session`</b>:  Requests session for HTTP requests. 
+
+
+
+
+---
+
+<a href="../src/github_client.py#L142"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+### <kbd>function</kbd> `delete_runner`
+
+```python
+delete_runner(path: Union[GithubOrg, GithubRepo], runner_id: int) → None
+```
+
+Delete the self-hosted runner from GitHub. 
+
+
+
+**Args:**
+
+ - <b>`path`</b>:  GitHub repository path in the format '<owner>/<repo>', or the GitHub organization  name. 
+ - <b>`runner_id`</b>:  Id of the runner. 
+
+---
+
+<a href="../src/github_client.py#L36"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+### <kbd>function</kbd> `get_runner_applications`
+
+```python
+get_runner_applications(
+    path: Union[GithubOrg, GithubRepo]
+) → List[RunnerApplication]
+```
+
+Get list of runner applications available for download. 
+
+
+
+**Args:**
+
+ - <b>`path`</b>:  GitHub repository path in the format '<owner>/<repo>', or the GitHub organization  name. 
+
+**Returns:**
+ List of runner applications. 
+
+---
+
+<a href="../src/github_client.py#L55"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+### <kbd>function</kbd> `get_runner_github_info`
+
+```python
+get_runner_github_info(
+    path: Union[GithubOrg, GithubRepo]
+) → list[SelfHostedRunner]
+```
+
+Get runner information on GitHub under a repo or org. 
+
+
+
+**Args:**
+
+ - <b>`path`</b>:  GitHub repository path in the format '<owner>/<repo>', or the GitHub organization  name. 
+
+
+
+**Returns:**
+ List of runner information. 
+
+---
+
+<a href="../src/github_client.py#L120"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+### <kbd>function</kbd> `get_runner_registration_token`
+
+```python
+get_runner_registration_token(path: Union[GithubOrg, GithubRepo]) → str
+```
+
+Get token from GitHub used for registering runners. 
+
+
+
+**Args:**
+
+ - <b>`path`</b>:  GitHub repository path in the format '<owner>/<repo>', or the GitHub organization  name. 
+
+
+
+**Returns:**
+ The registration token. 
+
+---
+
+<a href="../src/github_client.py#L102"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+### <kbd>function</kbd> `get_runner_remove_token`
+
+```python
+get_runner_remove_token(path: Union[GithubOrg, GithubRepo]) → str
+```
+
+Get token from GitHub used for removing runners. 
+
+
+
+**Returns:**
+  The removing token. 
+
+