Add optional rootless BuildKit sidecar to dynamic-rp chart#11882
Draft
willdavsmith wants to merge 3 commits into
Draft
Add optional rootless BuildKit sidecar to dynamic-rp chart#11882willdavsmith wants to merge 3 commits into
willdavsmith wants to merge 3 commits into
Conversation
Adds an optional, rootless BuildKit sidecar to the dynamic-rp Pod that in-cluster Terraform recipes can drive via the buildctl CLI to build and push container images. Motivating consumer is the new Radius.Compute/containerImages resource type in resource-types-contrib. Chart additions (deploy/Chart): - buildkitd sidecar container (rootless, no privileged Pod, no host Docker socket) with default-on enabled flag, configurable image, PSA mode (restricted/baseline), credentialsSecret, and resource limits/requests - buildctl-init init container that mounts the buildctl CLI into the dynamic-rp container's PATH - registry-credentials volume mounting an operator-supplied Secret at ~/.docker/config.json - RBAC: batch/jobs access for recipe-spawned Jobs - NOTES.txt warnings for misconfigured PSA mode and missing creds - helm-unittest cases covering buildkit sidecar shape (default-on / disabled) and a drift-guard against the install.go path contract Other changes: - pkg/recipes/terraform/install.go: log INFO when no pre-mounted Terraform binary is present, naming the expected paths - Fixes a path-mismatch bug in the chart's Terraform pre-mount init script (was writing to a directory the runtime never reads from) - New contributor doc covering the buildkit subsystem and the local-exec recipe pattern - Design doc copied to eng/design-notes/recipes/ Coordinates with resource-types-contrib PR for the resource type and recipe.
|
This PR requires exactly 1 of the following labels: pr:standard, pr:important. Label descriptions:
@willdavsmith, please add the appropriate label to this PR before merging. |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the deploy/Chart dynamic-rp deployment to optionally run a rootless BuildKit sidecar (default on) and adds supporting documentation/tests, enabling in-cluster image build/push scenarios without relying on a host Docker socket.
Changes:
- Add
dynamicrp.buildkit.*values and wire abuildkitdsidecar +buildctl-initinit container into the dynamic-rp Deployment. - Fix Terraform pre-mount pathing in the chart and add a drift-guard helm-unittest to keep chart/runtime paths aligned.
- Add operator-facing NOTES warnings plus new design/contributor docs for the subsystem.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/recipes/terraform/install.go | Logs an INFO message when no pre-mounted Terraform binary is present (to aid diagnosis). |
| eng/design-notes/recipes/2026-04-container-images-resource-type.md | Adds a design note for the containerImages resource type and how BuildKit is used. |
| docs/contributing/contributing-code/contributing-code-writing/buildkit-recipes.md | Documents the chart’s BuildKit sidecar and the recipe authoring pattern it enables. |
| deploy/Chart/values.yaml | Introduces the dynamicrp.buildkit values surface (enabled/psaMode/image/credentials/resources). |
| deploy/Chart/tests/helpers_test.yaml | Adds helm-unittest coverage for BuildKit enable/disable and Terraform-path drift guard. |
| deploy/Chart/templates/NOTES.txt | Adds install-time warnings for incompatible PSA mode / missing registry credentials. |
| deploy/Chart/templates/dynamic-rp/rbac.yaml | Grants dynamic-rp RBAC permissions for batch Jobs. |
| deploy/Chart/templates/dynamic-rp/deployment.yaml | Implements the terraform pre-mount fix and adds BuildKit containers/env/volumes. |
| @@ -185,6 +258,62 @@ spec: | |||
| {{- if .Values.dynamicrp.resources }} | |||
| resources:{{ toYaml .Values.rp.resources | nindent 10 }} | |||
| ┌─────────────────────────────────────────────────────────────┐ | ||
| │ dynamic-rp Pod │ | ||
| │ │ | ||
| │ buildkit-init ──► copies rootlesskit/buildkit data dir │ |
Comment on lines
+304
to
+310
| * **Provider**: `kreuzwerker/docker` ≥ 3.0. The provider speaks | ||
| BuildKit gRPC natively; it does not need a Docker CLI on the | ||
| recipe runner. | ||
| * **Endpoint**: configured via the `DOCKER_HOST` environment | ||
| variable, which dynamic-rp sets to | ||
| `unix:///run/buildkit/buildkit.sock` for recipe execution. The | ||
| recipe itself does not encode the endpoint. |
| runAsNonRoot: true | ||
| runAsUser: 65532 | ||
| {{- end }} | ||
| {{- end }} |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #11882 +/- ##
==========================================
+ Coverage 51.69% 51.72% +0.03%
==========================================
Files 725 726 +1
Lines 45595 45611 +16
==========================================
+ Hits 23570 23594 +24
+ Misses 19799 19793 -6
+ Partials 2226 2224 -2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
The containerImages recipe now reads registry credentials from a per-resource Radius.Security/secrets resource via the kubernetes_secret_v1 data source, matching the mysql pattern. The chart no longer needs to mount a Docker config.json — drop dynamicrp.buildkit.credentialsSecret value, volume, and volumeMount. Update fsGroup comment to reflect the buildctl binary mount (TCP, no socket sharing). Rewrite NOTES.txt to point platform engineers at the recipe-registration flow. Add a helm unittest covering the buildctl-init init container when terraform is disabled. Spelling list additions for new tech terms.
The previous default of `restricted` requires Kubernetes 1.30+ with the UserNamespacesSupport feature gate, which is not available out of the box on kind, k3d, Docker Desktop, or older managed clusters. This forced almost every operator trying out the BuildKit sidecar to immediately discover the failure mode and reinstall with --set dynamicrp.buildkit.psaMode=baseline. Flip the default so `rad install kubernetes` is a one-liner on every supported Kubernetes version. Operators who enforce PSA restricted cluster-wide and run a recent enough kernel can opt into the stricter sidecar profile with --set dynamicrp.buildkit.psaMode=restricted; the existing NOTES.txt preflight surfaces a clear remediation if it's selected on an incompatible cluster. Also update the NOTES.txt registry-credentials hint to reflect the PE-owned dockerconfigjson Secret model rather than the developer-owned Radius.Security/secrets language.
Radius functional test overviewClick here to see the test run details
Test Status⌛ Building Radius and pushing container images for functional tests... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional, rootless BuildKit sidecar to the
dynamic-rpPod, plus a small set of supporting fixes, so in-cluster Terraform recipes can build and push container images without a host Docker socket, a privileged Pod, or per-node host preparation.The motivating consumer is the new
Radius.Compute/containerImagesresource type — see companion PR radius-project/resource-types-contrib#151.What's in this PR
Helm chart (
deploy/Chart)buildkitdsidecar container (rootless, listens on Pod loopback TCP) added to the dynamic-rp Pod.buildctl-initinit container that copies thebuildctlCLI into a sharedemptyDir, mounted into the dynamic-rp container'sPATH.~/.docker/config.jsoninside the dynamic-rp container.dynamicrp.buildkit.*values surface:enabled(defaulttrue),psaMode(restricted/baseline),image,credentialsSecret, andresourceslimits/requests.batch/jobsaccess (recipe-spawned Jobs).pkg/recipes/terraform/install.gopath constants.Pre-mount path fix (latent bug)
The chart's existing Terraform pre-mount init script was writing the binary to a directory the runtime never reads from. Fixed alongside the chart rework.
Observability
pkg/recipes/terraform/install.go::ensureGlobalTerraformBinarynow logs an INFO line when no pre-mounted binary is present, naming the expected paths. Helps operators diagnose chart misconfiguration.Documentation
eng/design-notes/recipes/2026-04-container-images-resource-type.md.docs/contributing/contributing-code/contributing-code-writing/buildkit-recipes.mdcovering the buildkit subsystem and thelocal-execrecipe pattern.Coordination
Companion PR: radius-project/resource-types-contrib#151 (resource type + recipe). They should land together; this one is reviewable independently.
Design
Design review for this feature was completed prior to this PR.
Notable details
enabled: true. The buildkit sidecar runs by default on a fresh install. Operators who don't want it can--set dynamicrp.buildkit.enabled=false.psaMode=restrictedrequires Kubernetes ≥ 1.30 withUserNamespacesSupport(useshostUsers: false). Falls back tobaselinefor older clusters; the chart'sNOTES.txtwarns when the configured mode is incompatible.credentialsSecretis unset by default. Without it, builds run but pushes fail withunauthorized. NOTES.txt warns.deploy/Chart/tests/helpers_test.yamlasserts the chart still references the same Terraform pre-mount paths the runtime reads from. If you rename one, you'll be told to rename the other.Testing
helm-unittest74/74 pass (including 3 new tests).go test ./pkg/recipes/terraform/...passes.Follow-ups
Tracking separately so this PR stays focused:
dynamicrp.buildkit.enabled=trueand deploys acontainerImagesapp from the merged contrib recipe (Add full e2e CI lane for BuildKit sidecar + Radius.Compute/containerImages #11898).pkg/recipes/terraform/execute.gothat surfaces only on TF 1.15+ (currently masked by the 1.14.9 pin) (Recipe-context ordering bug in generateConfig / GetRecipeMetadata blocks Terraform 1.15+ unpin #11897).values.schema.jsonfor the chart (Introduce values.schema.json for the radius Helm chart #11899).