Repository to test the capabilities of GitHub action and how to manage them at scale.
GitHub Actions are manually, scheduled, or even drive triggered. Any event defined at Events that trigger workflows can trigger a workflow.
- starter-workflows These are the workflow files for helping people get started with GitHub Actions.
- Triggering a workflow
- inputs example
- Choosing GitHub-hosted runners
- shell
- Workflow Syntax
- Workflow Commands
- Caching dependencies to speed up workflows
- Artifacts
- Approving workflow runs from public forks
- Usage limits
- Default environment variables
- Contexts
- Expresions
On linux runners is possible to run Docker containers and run steps inside them.
On linux runners is possible to run Docker containers and use then as services.
Worlflows can use GitHub Actions as steps, these GitHub Actions come form the same repo, other repo, or a Docker image from any Docker registry.
An ephemeral GITHUB_TOKEN is is passed to the workflows if none is defined, it makes easy to access to GitHub repository, however you must define the permissions of that GITHUB_TOKEN explicitly to avoid surprises. By dafault it has only content read permission.
Like other CIs GitHub action has a feature to mask secrets in logs,
when the value of the secret is found in the logs GitHub Action change the text to ****,
however like in other CIs it is easy to by pass this protection midifiying the output in some way
to leak the password.
- create a user account with permissions to the services needed (e.g. Compute Engine admin)
- Create a Workload Identity Pools (IAM & admin/Workload Identity Pools)
- name: GitHub Actions OIDC token
- id: github-actions-oidc-token
- issuer: https://token.actions.githubusercontent.com
- Audiences: Default audience
- Attribute Mapping:
- google.subject: 'repo:kuisathaverat/actions_poc:ref:refs/heads/main'
- Add the service account to the Workload Identity Pools (IAM & admin/Workload Identity Pools/GitHub Action identity pool pool details)
In order to make some test, I have configured a HashiCorp Vault service on a VM in CGP.
First of all we have to launch a VM and install HashiCorp Vault,
for our test a e2-micro machine running Debian 10 is enough for our service.
To install vault we can follow one of this tutorials Web UI,
or DEploy Vault.
Try to configure everythin from the host conlose due the service does not have TSL enabled yet.
We need a firewall rule to allow inbound TCP traffic to the port 8200 of the VM.
The next step is to get a valid TSL certificate to run the server with TLS,
the easy way is to use certbot to issue a certificate from let's encrypt. You need a DNS name in order to issue the certificate so you can use <IP>.nip.io as a DNS name,
the .nip.io service will resolve the IP of your VM like a regular DNS.
When you have the certificate you can edit the configuration of Vault to use TSL.
ui = true
disable_mlock = true
storage "raft" {
path = "./vault/data"
node_id = "node1"
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = "false"
tls_cert_file = "/home/user/fullchain.pem"
tls_key_file = "/home/user/privkey.pem"
}
api_addr = "http://127.0.0.1:8200"
cluster_addr = "https://127.0.0.1:8201"
From this point, you can use the UI to create a kv v2 secret engine, and some secrets to get from the GitHub Actions. We have everything ready to start.
hashicorp/vault-action require to enable JWT authentication.
export VAULT_ADDR=http://127.0.0.1:8200
vault login
vault auth enable jwtWhen you have JWT authentication enabled you will need to configure the JWT with the following command
vault write auth/jwt/config \
oidc_discovery_url='https://token.actions.githubusercontent.com' \
bound_issuer='https://token.actions.githubusercontent.com'finally you have to create a JWT role,
the important parameter here is the bound_subject that match with the sub attribute in the GitHub action request
(see About security hardening with OpenID Connect and JWT with GitHub OIDC Tokens)
vault delete auth/jwt/role/githubactions
vault write auth/jwt/role/githubactions \
role_type='jwt' \
bound_subject='repo:kuisathaverat/actions_poc:ref:refs/heads/main' \
bound_audiences='https://github.com/kuisathaverat' \
user_claim='repository' \
policies='default'This is an alternative configuration that uses bound_claims with a wildcard value instead bound_subject to filter the requests.
Note that the payload is JSON due a bug with the maps in the commandline attributes
vault delete auth/jwt/role/githubactions
vault write auth/jwt/role/githubactions -<<EOF
{
"role_type": "jwt",
"bound_claims": {
"sub": "repo:kuisathaverat/*"
},
"bound_claims_type": "glob",
"bound_audiences": "https://github.com/kuisathaverat",
"user_claim": "repository"
}
EOF- OpenID Vault example
- About security hardening with OpenID Connect
- Configuring OpenID Connect in Google Cloud Platform
- Configuring OpenID Connect in HashiCorp Vault
notes:
- cubbyhole engine is not accesible from the action
- Vault path for the engine kv v2 are like /data/
It is possible to execute a set of steps based on a matrix of values, this use case is common to test something in different OS, or test a dependency on different versions.
Reusing code is important, GitHub actions allow to define workflows that you can call from other workflows. This feature has some limitation slike you can not call a reusable workflow from a reusable workflow, but it is flexible enough to cover most of the cases.
- Reusing workflows example called and caller
- Reusing workflows
- uses
Another way to exted GitHub action is to create GitHub Action, A gitHub Action could be something as simple as a Docker container or an script. In this PoC we have converted the logic we use to generate a Runner Token to a GitHub Action, we have made a Docker version and a composite version.
A Composite GitHub Actions and use other GitHub Actions, this make possible to bundle funcionality or wrap other GitHub Actions. In the composite example we have used a bash script, but is possible to use any shell or script language supported (bash, powershell, Python, Node.js, ...). Python is probably a good candidate, it is a good script language, with it has tons of libraries, and support for unit tests.
- Get Runner Token GitHub Action Composite version
- Get Runner Token GitHub Action Docker version
- Creating a Docker container action
- Creating a composite action
- Metadata syntax for GitHub Actions
It is possible to use self-hosted runners, these runners are any kind of VM/machine/pod/... that can run the runner app. It is easy to provision a runner inside the same workflow you are running and deprovision it at the end of the workflow, also there are several implementations to provision runners on diferent cloud providers based on a GitHub app to request the provision/deprovision. In the PoC we have implemented 3 ways to deploy a runner:
- On demand GCP runner example
- On demand GKE runner example
- On demand GCP runner example ussing an Action
- Runner Docker container
The three ways use the workflow pattern provision->build->deprovision,
this approacha can apply to any Cloud provider,
To provision operation you can use a CLI tool, Terraform, or other way to provision resources.
In the k8s example, we use a Docker container with the runner installed along witha a k8s manifest file to define runner pod.
To make the deploy kubeclt is the tools used to provision and deprovision the pod.
- About self-hosted runners
- actions-runner-controller This controller operates self-hosted runners for GitHub Actions on your Kubernetes cluster.
- terraform-aws-github-runner This Terraform module creates the required infrastructure needed to host GitHub Actions self-hosted, auto-scaling runners on AWS spot instances.
- Terraform / Packer project for scalable self hosted GitHub action runners on GCP This project leverages Terraform and Packer to deploy and maintain scalable self hosted GitHub actions runners infrastructure on GCP for a GitHub organization.
- gce-github-runner Ephemeral GCE GitHub self-hosted runner.
- orka-actions-up Run self-hosted, macOS workflows on MacStadium's Orka.
It is possiblt tu run a Docker container as a service and use it, so I have tried to start a Jenkins container configurated with JCasC, and lauch a job created with jobDSL. The result is nice, you can launch Jenkins configure the job with jobDSL, then launch the job and retrieve the logs.
The GitHub Provider allows to use Terraform to manage a GitHub organization and all its resources. GitHub Provider](https://registry.terraform.io/providers/integrations/github/latest/docs) can create and configure repositories, teams, Webhooks, Projects, issues, secrets, GPG Keys, SSH Keys,... everything defined in Terraform files with all the advantages it has like repeability and manteniability.
- Simple
- Zero learn curve
- Marketplace with a huge action alredy implemented
- Strong comunnity
- Overall well Documented
- GitHub integration
- Security managed along with GitHub access
- Linux/Windows/masOS runners support
- Easy tooling manage (Go, Node.js, Java, Docker,...)
- GCP/Azure/AWS/Vault OpenID integration
- ONPrem runners support
- Kubernetes runners implemented
- AWS runners implemented
- Orka runners implemented
- Cache support for the most common build systems
- Reusable blocks of code (library of action/workflows)
- Library code support unit tests
- Matrix jobs execution native support
- Container services support
- Public/private Artifact repository (GitHub packages)
- Public/private Docker registry (GitHub packages)
- Mask secrets support
- Logs messages group support
- Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests
- Skipping workflow runs
- Deploying to Google Kubernetes Engine
- Working with the Container registry
- Connecting a repository to a user-owned package on GitHub
- Upgrading a workflow that accesses ghcr.io
- Migrating from Jenkins to GitHub Actions
- Adding a workflow status badge
- Enabling debug logging
- Notifications for workflow runs
- GitHub Actions self-hosted runners on Google Cloud
- Security hardening for GitHub Actions
- github-script Write workflows scripting the GitHub API in JavaScript
- file-changes-action get outputs of all of the files that have changed in your repository
- cache Cache dependencies and build outputs in GitHub Actions
- docker/metadata-action GitHub Action to extract metadata from Git reference and GitHub events.
- docker/build-push-action GitHub Action to build and push Docker images with Buildx with full support of the features provided by Moby BuildKit builder toolkit.
- docker/setup-qemu-action GitHub Action to install QEMU static binaries.
- setup-buildx-action GitHub Action to set up Docker Buildx.
- docker/login-action GitHub Action to login against a Docker registry.
- create-or-update-comment A GitHub action to create or update an issue or pull request comment.
- actions/stale Warns and then closes issues and PRs that have had no activity for a specified amount of time.
- labeler Automatically adds or removes labels from issues, pull requests and project cards. Actions on your Kubernetes cluster.
- ossf-scorecard-action Official GitHub Action for OSSF Scorecards.
- rotate-gcp-service-account-keys This action rotates GCP service account keys.
- google-github-actions/get-gke-credentials This action configures authentication to a GKE cluster via a kubeconfig file that can be used with kubectl or other methods of interacting with the cluster.
- actions/snyk A set of GitHub Action for using Snyk to check for vulnerabilities in your GitHub projects.
- actions/kubernetes-security-config-watch This Git Action run security lint check against Kubernetes workloads in Git workflow (PR open, commit pushed etc.).
- actions/hashicorp-setup-terraform The hashicorp/setup-terraform action is a JavaScript action that sets up Terraform CL.
- actions/packer-github-actions GitHub Action for running Packer commands.
- actions/slack-send Send data into Slack using this GitHub Action!
- actions/jenkinsfile-runner-prepackaged This is a POC how to run Jenkinsfiles inside GitHub Actions.
- set-up-wireguard This GitHub action sets up a WireGuard connection in your workflow run.
- connect-vpn This action is a connect ovpn script