Update documentation

docs/explanation/charm-architecture.md

@@ -1,18 +1,74 @@
 # Charm architecture
 
-A [Juju](https://juju.is/) [charm](https://juju.is/docs/olm/charmed-operators) to operate a set of [GitHub self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners) while managing the security and resource usage.
+A [Juju](https://juju.is/) [charm](https://juju.is/docs/olm/charmed-operators) to operate a set of [GitHub self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners) while managing security and resource usage.
 
 Conceptually, the charm can be divided into the following:
 
-- Management of LXD ephemeral virtual machines to host [ephemeral self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/autoscaling-with-self-hosted-runners#using-ephemeral-runners-for-autoscaling).
-- Management of a [Python web service for checking GitHub repository settings](https://github.com/canonical/repo-policy-compliance).
+- Management of LXD ephemeral virtual machines to host [ephemeral self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/autoscaling-with-self-hosted-runners#using-ephemeral-runners-for-autoscaling)
+- Management of the network
+- GitHub API usage
+- Management of [Python web service for checking GitHub repository settings](https://github.com/canonical/repo-policy-compliance)
+- Management of dependencies
 
 ## LXD ephemeral virtual machines
 
- To ensure a clean and isolated environment for every runner, Self-hosted runners are hosted using LXD virtual machines. The charm spawns virtual machines setting resources based on charm configurations. The self-hosted runners start with the ephemeral option and will clean themselves up once the execution has finishes, freeing the resources.
+To ensure a clean and isolated environment for every runner, self-hosted runners use LXD virtual machines. The charm spawns virtual machines, setting resources based on charm configurations. The self-hosted runners start with the ephemeral option and will clean themselves up once the execution has finished, freeing the resources. This is [similar to how GitHub hosts their runners due to security concerns](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#self-hosted-runner-security). 
+
+As the virtual machines are single-use, the charm will replenish virtual machines on a regular schedule. This time period is determined by the [`reconcile-interval` configuration](https://charmhub.io/github-runner/configure#reconcile-interval).
+
+On schedule or upon configuration change, the charm performs a reconcile to ensure the number of runners managed by the charm matches the [`virtual-machines` configuration](https://charmhub.io/github-runner/configure#virtual-machines), and the resources used by the runners match the various resource configurations.
+
+The virtual machines hosting the runner use random access memory as disk; therefore, the [`vm-disk` configuration](https://charmhub.io/github-runner/configure#vm-disk) can impact the memory usage of the Juju machine. This is done to prevent disk IO exhaustion on the Juju machine on disk-intensive GitHub workflows. In the future, an alternative method to prevent disk IO exhaustion will be implemented.
+
+## Network configuration
+
+The charm respects the HTTP(S) proxy configuration of the model configuration of Juju. The configuration can be set with [`juju model-config`](https://juju.is/docs/juju/juju-model-config) using the following keys: `juju-http-proxy`, `juju-https-proxy`, `juju-no-proxy`. The GitHub self-hosted runner applications are configured to use the proxy configuration.
+
+If an HTTP(S) proxy is used, all HTTP(S) requests in the GitHub workflow will be transparently routed to the proxy with [aproxy](https://github.com/canonical/aproxy). Iptables are set up to route network traffic to the destination on ports 80 and 443 to the aproxy. The aproxy will route received packets to the configured HTTP(S) proxy. The service is installed on each runner virtual machine and configured according to the proxy configuration from the Juju model.
+
+The nftables on the Juju machine are configured to deny traffic from the runner virtual machine to IPs on the [`denylist` configuration](https://charmhub.io/github-runner/configure#denylist). The runner will always have access to essential services such as DHCP and DNS, regardless of the denylist configuration.
+
+
+## GitHub API usage
+
+The charm requires a GitHub personal access token for the [`token` configuration](https://charmhub.io/github-runner/configure#token). This token is used for:
+
+- Requesting self-hosted runner registration tokens
+- Requesting self-hosted runner removal tokens
+- Requesting a list of runner applications
+- Requesting a list of self-hosted runners configured in an organization or repository
+- Deletion of self-hosted runners
+
+The token is also passed to [repo-policy-compliance](https://github.com/canonical/repo-policy-compliance) to access GitHub API for the service.
 
 ## GitHub repository setting check
 
-A [flask application](https://flask.palletsprojects.com/) hosted on [gunicorn](https://gunicorn.org/) provides a RESTful HTTP API to check the settings of GitHub repository. This ensures the GitHub repository settings do not allow the execution of code not reviewed by maintainers on the self-hosted runners.
+The [repo-policy-compliance](https://github.com/canonical/repo-policy-compliance) is a [Flask application](https://flask.palletsprojects.com/) hosted on [Gunicorn](https://gunicorn.org/) that provides a RESTful HTTP API to check the settings of GitHub repositories. This ensures the GitHub repository settings do not allow the execution of code not reviewed by maintainers on the self-hosted runners.
+
+Using the [pre-job script](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/running-scripts-before-or-after-a-job#about-pre--and-post-job-scripts), the self-hosted runners call the Python web service to check if the GitHub repository settings for the job are compliant. If not compliant, it will output an error message and force stop the runner to prevent code from being executed.
+
+## Dependencies management
+
+Upon installing or upgrading the charm, the kernel will be upgraded, and the Juju machine will be restarted if needed.
+
+The charm installs the following dependencies:
+
+- For running repo-policy-compliance
+  - gunicorn
+- For firewall to prevent runners from accessing web service on the denylist
+  - nftables
+- For virtualization and virtual machine management
+  - lxd
+  - cpu-checker
+  - libvirt-clients
+  - libvirt-daemon-driver-qemu
+  - apparmor-utils
+
+These dependencies can be regularly updated using the [landscape-client charm](https://charmhub.io/landscape-client).
+
+The charm installs the following dependencies and regularly updates them:
+
+- repo-policy-compliance
+- GitHub self-hosted runner application
 
-Using the [pre-job script](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/running-scripts-before-or-after-a-job#about-pre--and-post-job-scripts), the self-hosted runners calls the Python web service to check if the GitHub repository settings for the job are compliant. If not compliant, it will output an error message and force stop the runner to prevent code from being executed.
+The charm checks if the installed versions are the latest and performs upgrades if needed before creating new virtual machines for runners.

docs/how-to/change-path.md

@@ -2,4 +2,8 @@
 
 This charm supports changing the GitHub repository or GitHub organization the self-hosted runners are connected to.
 
-By using [`juju config`](https://juju.is/docs/juju/juju-config) to change the [charm configuration path](https://charmhub.io/github-runner/configure#path) to another repository or organization, the charm unregisters and removes the old self-hosted runners and instantiates new ones for the new configuration.
+By using [`juju config`](https://juju.is/docs/juju/juju-config) to change the [charm configuration path](https://charmhub.io/github-runner/configure#path) to another repository or organization, the charm unregisters and removes the old self-hosted runners and instantiates new ones for the new configuration.
+
+```shell
+juju config <APP_NAME> path=<PATH>
+```

docs/how-to/change-token.md

@@ -1,18 +1,22 @@
 # How to change GitHub personal access token
 
-This charm supports changing [GitHub personal access token (PAT)](https://github.com/settings/tokens) used.
+This charm supports changing the [GitHub personal access token (PAT)](https://github.com/settings/tokens) used.
 
 ## Personal access token scope
 
-For using this charm for GitHub repository the following scopes should be selected:
+To use this charm for GitHub repositories, the following scopes should be selected:
 
 - `repo`
 
-For using this charm for GitHub organization the following scopes should be selected:
+To use this charm for GitHub organisations, the following scopes should be selected:
 
 - `repo`
 - `admin:org`
 
 ## Changing the token
 
-By using [`juju config`](https://juju.is/docs/juju/juju-config) to change the [charm configuration token](https://charmhub.io/github-runner/configure#token) the charm unregisters and removes the old self-hosted runners and instantiates new ones.
+By using [`juju config`](https://juju.is/docs/juju/juju-config) to change the [charm configuration token](https://charmhub.io/github-runner/configure#token) the charm unregisters and removes the old self-hosted runners and instantiates new ones.
+
+```shell
+juju config <APP_NAME> token=<TOKEN>
+```

docs/how-to/configure-denylist.md

@@ -0,0 +1,24 @@
+# How to restrict self-hosted runner network access
+
+The [`denylist` configuration](https://charmhub.io/github-runner/configure#denylist) can be used to restrict network access for self-hosted runners.
+
+This can be employed to prevent self-hosted runners from accessing the network on the Juju machine. Generally, all IPv4 local addresses should be included in the denylist:
+
+- 0.0.0.0/8
+- 10.0.0.0/8
+- 100.64.0.0/10
+- 127.0.0.0/8
+- 169.254.0.0/16
+- 172.16.0.0/12
+- 192.0.0.0/24
+- 192.0.2.0/24
+- 192.88.99.0/24
+- 192.168.0.0/16
+- 198.18.0.0/15
+- 198.51.100.0/24
+- 203.0.113.0/24
+- 224.0.0.0/4
+- 233.252.0.0/24
+- 240.0.0.0/4
+
+Additionally, include any IPv4 address or CIDR block that the runner should not have access to on the denylist.

docs/how-to/contribute.md

@@ -29,4 +29,4 @@ To test the charm, unit test can be ran with `tox -e unit` and the integration t
 
 ## Canonical Contributor Agreement
 
-Canonical welcomes contributions to the GitHub Runner Operator. Please check out our [contributor agreement](https://ubuntu.com/legal/contributors) if you’re interested in contributing to the solution.
+Canonical welcomes contributions to the GitHub Runner Operator. Please check out our [contributor agreement](https://ubuntu.com/legal/contributors) if you’re interested in contributing to the solution.

docs/how-to/repo-policy.md

@@ -0,0 +1,21 @@
+# How to comply with repository policies
+
+The charm enforces a set of best practice GitHub repository settings. Self-hosted runners managed by the charm will not run jobs on repositories not compliant with the practices. This will be opt-in in the future.
+
+The repository settings are enforced with this [Python library](https://github.com/canonical/repo-policy-compliance). The rules enforced are different depending on how the GitHub Actions workflow is triggered. The details can be found in the README.
+
+In this guide, a recommended set of policies will be presented, but any set repository settings that passes the [Python library](https://github.com/canonical/repo-policy-compliance) checks will work with the self-hosted runners managed by this charm.
+
+## Recommended policy
+
+- For outside collaborators the permission should be set to read. See [here](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/managing-repository-settings/managing-teams-and-people-with-access-to-your-repository#changing-permissions-for-a-team-or-person) for instructions to change collaborator permissions. Outside collaborators will still be able to contribute with pull requests, but reviews will be needed. Details in a later section.
+- Create the following branch protection rules, with the instructions [here](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/managing-a-branch-protection-rule#creating-a-branch-protection-rule):
+  - branch name pattern matching only the default branch of the repository, such as `main`, with the follow enabled:
+    - `Dismiss stale pull request approvals when new commits are pushed`
+    - `Do not allow bypassing the above settings`
+
+With these settings, the common workflow of creating branches with pull requests and merging to the default branch is supported. Other GitHub Actions workflow triggers such as workflow_dispatch, push, and schedule are supported as well.
+
+### Working with outside collaborators
+
+Contributions from outside collaborators (in the case where a repository is public) need to be handled slightly differently. As such, this charm requires pull requests by outside collaborators to be reviewed by someone with `write` permission or above. Once the review is completed, the reviewer should add a comment including the following string: `/canonical/self-hosted-runners/run-workflows <commit SHA>`, where `<commit SHA>` is the commit SHA of the approved commit. Once posted, the self-hosted runners will run the workflow for this commit.

docs/how-to/run-on-lxd.md

@@ -2,12 +2,12 @@
 
 This machine charm needs to run on virtual machines with nested virtualization enabled.
 
-By default, juju machine on LXD are containers.
+By default, juju machines on LXD are containers.
 
 To run this charm on LXD, add `virt-type=virtual-machine` to the constraints during deployment:
 
 ```shell
-juju deploy github-runner --constraints="cores=4 mem=16G virt-type=virtual-machine" --config token=<TOKEN> --config path=<OWNER/REPO>
+juju deploy github-runner --constraints="cores=2 mem=16G virt-type=virtual-machine" --config token=<TOKEN> --config path=<OWNER/REPO>
 ```
 
-This constraint ensure the juju machine hosting the charm is a LXD virtual machine.
+This constraint ensures the juju machine hosting the charm is a LXD virtual machine. See [Managing resource usage](https://charmhub.io/github-runner/docs/managing-resource-usage) for recommendation on `cores` and `mem` constraint.

docs/index.md

@@ -1,21 +1,25 @@
 A [Juju](https://juju.is/) [charm](https://juju.is/docs/olm/charmed-operators) for deploying and managing [GitHub self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners) on virtual machines.
 
-This charm simplifies initial deployment and "day N" operations of GitHub self-hosted runners. The charm makes it easy to manage self-hosted runners with security and hardware resource usage in mind.
+This charm simplifies the initial deployment and "day N" operations of GitHub self-hosted runners. The charm makes it easy to manage self-hosted runners with security and hardware resource usage in mind.
 
 Operating a self-hosted runner comes with [certain security concerns according to GitHub](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#self-hosted-runner-security).
-Just like GitHub's, the self-hosted runners managed by the charm are isolated in a single-use virtual machine. However, arbitrary code execution is possible under certain repository settings. This can be leveraged by malicious actors in a number of ways, such as, crypto-mining. To combat this, the charm enforces a set of GitHub repository settings to ensure the executed code is reviewed by someone trusted.
+Just like GitHub's, the self-hosted runners managed by the charm are isolated in a single-use virtual machine.
 
-The charm also upgrades dependencies on a schedule to mitigate security risks. The upgrade includes linux kernel upgrades, automatically rebooting the machines. This ensures the latest security patches are installed within minutes.
+Some of the charm dependencies upgrades on a schedule to migrate security risks. The landscape-client charm can be deployed with this charm to ensure other dependencies are up to date.
 
 The charm maintains a set of ephemeral self-hosted runners, each isolated in a single-use virtual machine instance. To prevent disk IO exhaustion, random access memory is used as disk for the virtual machine instances. In addition, resource limits for the self-hosted runners can be configured.
 
+See [charm architecture](https://charmhub.io/github-runner/docs/charm-architecture) for more information.
+
 This charm will make operating GitHub self-hosted runners simple and straightforward for DevOps or SRE teams through Juju's clean interface.
 
+The charm enforces a set of GitHub repository settings as best practice. This is planned to be opt-in in the future. See [How to comply with repository policies](https://charmhub.io/github-runner/docs/repo-policy).
+
 ## In this documentation
 
 | | |
 |--|--|
-|  [Tutorials](https://charmhub.io/github-runner/docs/quick-start)</br>  Get started - a hands-on introduction to using the GitHub runner charm for new users </br> | [How-to guides](https://charmhub.io/github-runner/docs/how-to-comply-security) </br> Step-by-step guides covering key operations and common tasks |
+|  [Tutorials](https://charmhub.io/github-runner/docs/quick-start)</br>  Get started - a hands-on introduction to using the GitHub runner charm for new users </br> | [How-to guides](https://charmhub.io/github-runner/docs/run-on-lxd) </br> Step-by-step guides covering key operations and common tasks |
 | [Reference](https://charmhub.io/github-runner/docs/actions) </br> Technical information - specifications, APIs, architecture | [Explanation](https://charmhub.io/github-runner/docs/charm-architecture) </br> Concepts - discussion and clarification of key topics  |
 
 ## Contributing to this documentation
@@ -34,20 +38,3 @@ The GitHub runner charm is a member of the Ubuntu family. It's an open-source pr
 - [Contribute](Contribute)
 
 Thinking about using the GitHub runner charm for your next project? [Get in touch](https://chat.charmhub.io/charmhub/channels/charm-dev)!
-
-# Contents
-
-1. [Tutorial](tutorial)
-  1. [Quick start](tutorial/quick-start.md)
-  1. [Managing resource usage](tutorial/managing-resource-usage.md)
-1. [How to](how-to)
-  1. [How to run on LXD cloud](how-to/run-on-lxd.md)
-  1. [How to comply with security requirements](how-to/comply-security.md)
-  1. [How to change GitHub personal access token](how-to/change-token.md)
-  1. [How to change repository or organization](how-to/change-path.md)
-  1. [How to contribute](how-to/contribute.md)
-1. [Reference](reference)
-  1. [Actions](reference/actions.md)
-  1. [Configurations](reference/configurations.md)
-1. [Explanation](explanation)
-  1. [Charm architecture](explanation/charm-architecture.md)

docs/reference/actions.md

@@ -1,3 +1,3 @@
 # Actions
 
-See [Actions](https://charmhub.io/github-runner/actions).
+See [Actions](https://charmhub.io/github-runner/actions).

docs/reference/configurations.md

@@ -1,3 +1,3 @@
 # Configurations
 
-See [Configure](https://charmhub.io/github-runner/configure).
+See [Configure](https://charmhub.io/github-runner/configure).

docs/tutorial/managing-resource-usage.md

@@ -24,4 +24,4 @@ The recommended combined resource usage is:
 
 ## Juju machine constraints
 
-During [deployment of the charm](https://juju.is/docs/juju/juju-deploy), constraints can be used to specify the juju machine resource requirements. For example, `juju deploy github-runner --constraints="cores=4 mem=16G disk=20G"`.
+During [deployment of the charm](https://juju.is/docs/juju/juju-deploy), constraints can be used to specify the juju machine resource requirements. For example, `juju deploy github-runner --constraints="cores=4 mem=16G disk=20G"`.