Skip to content

v1.4.0

Latest
Latest

Choose a tag to compare

View all tags
@karlkfi karlkfi released this 10 Aug 04:03
· 4 commits to main since this release
v1.4.0
747bb85
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Zero-idle self-hosted GitHub Actions runners for multi-tenant Kubernetes.

Warning

One required upgrade step, and nothing in this release is breaking. Apply the chart's CRDs before upgrading, or four new wire fields will not exist. Every other change here defaults to the previous behaviour. Read Upgrading below.

Highlights

Cross-namespace EgressProxy sharing, actually enforced. sharing.allowedNamespaces was served in the v2beta1 API with nothing honouring it, so an operator could set a field that did nothing. This release builds the path and guards it: a consent check on the providing EgressProxy, CA distribution, and dual-side NetworkPolicy. Absent or empty sharing denies, so the pre-existing same-namespace posture stays the default and the unset case (#1347).

A curated runner template library. Templates CI already validates ship as a kustomize base you can patch: a plain baseline plus the kata-dind and privileged-dind templates the dogfood e2e suite exercises. No new CRD. The constraint that only CI-exercised templates ship is a gate rather than a convention: make template-library-check reconciles the shipped and exercised sets both ways, and every entry is admitted by a real apiserver on each integration run (#1344).

An abandoned job's run is force-cancelled and re-run automatically, on both acquisition tiers. A worker reaped before its container ever started used to leave GitHub waiting out its ~15-minute unstarted-job timeout. The run is now cancelled in about a second, then re-run once the owner places a worker pod again. Shipped classic-only, then ported to the scale-set tier inside this same release, so no tag ever published the asymmetry (#1298, #1336, #1366).

Worker-capacity conditions are Prometheus gauges on v2 RunnerSets. Four new gauges mirror the conditions a v2 RunnerSet already reported, so quota pressure, unschedulable workers and a declined capacity gate are alertable without reading conditions. WorkerCapacityDeclined carries its reason as a label (#1200, #1225).

Note

Job duration and pod-creation latency are still classic-tier only. actions_gateway_job_duration_seconds and actions_gateway_pod_creation_latency_seconds are observed only on the classic acquisition path, so on a ScaleSet set, the default tier, they are absent, and the two ActionsGatewayPodCreationLatency alerts, four recording rules and their dashboard panels read blank. A silent latency alert there is a missing series, not a met SLO. This is not new in 1.4; it predates v1.3.0. Tracked as Q713.

Upgrading

One step, and it is the same step every upgrade of this chart needs. Full procedure in the upgrade guide, which documents rollback per component: GMC, AGC, proxy, and worker image.

1. Apply the chart's CRDs. Part of every upgrade of this chart, not a one-off: Helm installs the chart-root crds/ directory on a fresh install only.

helm show crds oci://ghcr.io/actions-gateway/charts/actions-gateway \
  --version 1.4.0 | kubectl apply -f -

Charts carry no leading v. Images are tagged v1.4.0, charts 1.4.0.

Three changes that need no action, but will be visible.

  • An abandoned run is force-cancelled and re-run automatically — a job whose worker was reaped before it ran now concludes cancelled in about a second instead of at GitHub's timeout, and is re-run when capacity returns. Both tiers (#1298, #1336, #1366).
  • v1alpha1 now warns on every apply — deprecated, not removed; see Deprecations below (#1199).
  • allowedInfraPriorityClasses is new and needs no action unless you set it. The PriorityClassAllowlist CR gained a second list for infrastructure classes, sourced from the watched CR so an admin can add one without a GMC rollout. Unset or empty forbids every named class, so the secure posture is the unset case (#1198).

Deprecations

Nothing is removed in this release. Both notices below point at v2.0.0, unchanged from v1.3.0 except that one of them now warns.

v1alpha1 is deprecated, and as of this release the apiserver warns on every apply (#1199). That closes the asymmetry v1.3.0's notes flagged: v2alpha1 warned and v1alpha1 did not, so nothing reminded an operator it was going away. Tenants still on v1alpha1 migrate with gag-migrate: Migrating a tenant to v2.

v2alpha1 and Classic are removed at v2.0.0 too. v2alpha1 has warned since v1.3.0. Moving to v2beta1 is a re-apply rather than a migration tool: Before you upgrade past v2.0.0.

Everything since v1.3.0

192 commits. These 16 change the shipped binaries and chart, and two more change the reference dashboards operators apply; build, CI, test, tooling, and docs work is left out, and is summarised under Project and tooling below.

New features (10)

Cross-namespace egress proxy sharing

  • Enforce cross-namespace EgressProxy sharing, v2 API M4 (#1347)

Runner template library

  • Ship a curated runner template library as a kustomize base (#1344)

Abandoned-run recovery

  • Force-cancel an abandoned job's run for a ~1s honest ending (#1298)
  • Auto re-run a force-cancelled abandoned run when capacity returns (#1336)
  • Port the force-cancel and auto re-run to the scale-set tier (#1366)

Observability

  • Export v2 RunnerSet worker-capacity conditions as gauges (#1200)
  • Gauge WorkerCapacityDeclined with its reason label (#1225)

Admission and API

Fixes (6)

Job conclusions and the scale-set listener

  • A never-run job must not conclude its run green (#1277)
  • A worker removed before it ran is reported abandoned, and its assignment released (#1197)
  • Read the queue's outstanding conclusions on the way out (#1308)
  • Bound the scale-set listener's replay guards to the queue (#1196)
  • DeleteMessage reports whether the wire deleted anything (#1193)

Reconcile

Shipped dashboards (2)

The reference Grafana dashboards under deploy/monitoring/ are applied by operators, so changes to them are user-visible even though they are in no image.

API and metric surface

Four wire fields, one condition reason, and six metrics are published for the first time. No field, enum value, default, metric, or Event reason was removed or renamed, and no enum constraint or default changed. Field descriptions are in the generated API reference, which covers v2beta1; a deprecated-version field still needs kubectl explain.

New spec fields (4)

PriorityClassAllowlist

RunnerSet

  • spec.proxyRef: the EgressProxy this set uses. Optional, because direct egress is a defined behaviour rather than a failure (#1347)

ActionsGateway

ProxyObjectRef (carried by both refs above)

  • namespace: empty means the referrer's own namespace, so the same-namespace posture is the default and the unset case (#1347)
New condition reasons (1)
New metrics (6), and one label added to two existing ones

All prefixed actions_gateway_, listed here without it. Remedies and alerting guidance are in observability-metrics.

Abandoned-run recovery

v2 RunnerSet worker capacity

One label added. Both abandoned-run counters gained a tier label (classic, scaleset) when the recovery was ported (#1366). An existing query that does not group by it keeps its meaning; one that aggregates across tiers now sums both. The shipped dashboards were updated to split on it.

No new Kubernetes Event reasons, and none removed: this release reuses the existing 24.

Validation

Both release candidates produced a passing verdict, and no leg failed in either. v1.4.0-rc.1 passed, then the scale-set port (#1366) and a docs sweep landed after its commit, so rc.1 stopped being the candidate: a candidate's pass does not transfer to a later one carrying new product code. v1.4.0-rc.2 was cut and validated in full.

Scoped to what the record supports: validate-release.sh landed the day v1.1.0 was tagged, so v1.0.0 and v1.1.0 predate it, and no plan doc records a run for v1.2.0. The 1.3 line is the only prior line with a recorded validation history, and it needed four candidates before one returned a verdict at all. So this is the first line in which no candidate failed a leg, not the first time that has ever happened.

74/74 e2e specs on live GKE, 62 ok and 0 failed: the four legs, and what none of them assert

Run against the signed release artifacts rather than a build (run), 26m59s end to end:

  • The full e2e matrix on gateway-provisioned runners, 74/74 specs, 62 ok, 0 failed, 12 skipped
  • Both sizing profiles actuating on real worker pods, with the derived value confirmed at the pod: 1500m where the templates asked for 2 and 3 CPU, and Throughput Active on 176 samples
  • The signed v2 CRD artifact downloaded, signature-verified, applied, and all five CRDs registered
  • Teardown back to zero nodes, confirmed against the compute API rather than the gate's own teardown line

That is what the gate asserts. It does not assert cross-namespace proxy sharing, the capacity gate, the namespace-quota rung, or anything GHES. The run targets github.com and builds an autoscaling cluster with headroom rather than a constrained one. Those rest on the unit, envtest, and e2e suites instead.

Artifact verification cleared separately: 7/7 OCI signatures across five images and both charts, the verify-blob signature on the v2 CRD manifest, and a build-provenance attestation whose signer URI ends publish.yml@refs/tags/v1.4.0-rc.2 over the tagged commit. Each green check was re-run against a deliberately wrong identity and each failed, so the passes discriminate rather than merely exiting 0.

Full changelog: v1.3.0...v1.4.0

Project and tooling

Two operator guides are new. The runner template library is linked from its feature above. Personas is reference material with no feature of its own: it names the five roles the operator docs are written for, so a page can say who it is for instead of assuming. Troubleshooting and security operations both grew substantially.

The rest is contributor-facing, listed because a release is also a claim about how the project is built.

Release, CI, and repository tooling

Release process

  • Report the semver floor the merged work already forces (#1313)
  • Narrow the semver floor past comment-only diffs (#1341)
  • Reconcile the roadmap's release gates against the backlog labels (#1367)

The 1.4 pre-flight sweep

A deliberate pass over the marketing and operator surfaces before the tag, which is where several of this release's docs corrections came from. It found an executive summary promising OOM-killed jobs are re-run when the provisioner deliberately excludes them, a worst-case recovery figure quoted as the typical case, and two comparison rows that had gone false at datable upstream releases. It also found the abandoned-run docs disagreeing with each other about tier scope, which is what surfaced the gap #1366 then closed.

Security

No security advisory accompanies this release, and no fix here is embargoed or backported.

Two changes harden trust and isolation rather than patching a reported vulnerability. Cross-namespace EgressProxy sharing (#1347) fails closed on every axis: absent or empty sharing denies, the AGC cannot read a remote EgressProxy at all so the GMC mediates rather than widening tenant RBAC, and both sides get a NetworkPolicy. The infra PriorityClass allowlist (#1198) keeps its secure posture as the unset case.

Every pull request, with no path filter deliberately, runs govulncheck across the workspace and a Trivy scan of the images, and main re-runs both on every push.

Container images

Pin these immutable multi-arch index digests rather than the floating v1.4.0 tag. Each index serves both linux/amd64 and linux/arm64, so one pinned digest schedules on either.

  • gmc: ghcr.io/actions-gateway/gmc@sha256:6da59e421839b022b4605979b6295e535574ae0bc2be8420255eb4a8d27a7570
  • agc: ghcr.io/actions-gateway/agc@sha256:ecfbb75ec912a887cc1a9c640dab85a1b44fdbb9f01d071521ab544d7b4742c8
  • proxy: ghcr.io/actions-gateway/proxy@sha256:b53add9ae5d2f02c30f60b88a2ff3018ffbbd2d06aa41e7ef0ea90973db3bb99
  • worker: ghcr.io/actions-gateway/worker@sha256:8a6535a4f52fa8784020285a7f8177e03aef6bffa168c14dc47d9f839f886fba
  • wrapper: ghcr.io/actions-gateway/wrapper@sha256:99e8b1a5044d9e7342b193ab9347d9fbc07c27e9b84c2029aadd7d8ac244e91a

The chart's digest fields ship empty on purpose, so an unconfigured install fails closed rather than running a floating tag. These are the values to put in them.

Verifying this release

We sign every image and both charts.

make verify-release VERSION=v1.4.0

The signed v2 CRD manifest and SHA256SUMS ship as release assets with detached cosign bundles, and the publish workflow attests build provenance.

Assets 11
Loading