Skip to content

runtime: decode BuildKit aux trace records for per-vertex events#51

Merged
bilby91 merged 2 commits into
mainfrom
buildkit-trace-decoder
May 13, 2026
Merged

runtime: decode BuildKit aux trace records for per-vertex events#51
bilby91 merged 2 commits into
mainfrom
buildkit-trace-decoder

Conversation

@bilby91
Copy link
Copy Markdown
Member

@bilby91 bilby91 commented May 13, 2026

Summary

  • streamBuildOutput now routes moby.buildkit.trace aux records to a hand-rolled wire-format decoder (protowire) that parses the subset of controlapi.StatusResponse we need: Vertex.{name,cached,started,completed,error} and VertexLog.msg.
  • Emits BuildLayerEvent on per-digest start/complete transitions and BuildLogEvent per RUN log line. Closes consumers' BuildStart → silence → BuildCompleted gap under BuildKit.
  • Avoids pulling github.com/moby/buildkit as a direct dep (+253 transitive modules: containerd, k8s, sigstore, AWS/Azure SDKs). Net cost: +2 modules (golang/protobuf v1 shim, xerrors) from promoting google.golang.org/protobuf to direct.

Closes part of #50 (limitation 1, per-vertex events). Limitations 2 (extractBaseImages parser blind spots) and 3 (tarDirectory / .dockerignore) remain open.

Test plan

  • go test -race ./... — passes
  • go vet ./... — clean
  • make test-integration (20 tests) — all pass, 230s
  • New unit tests in runtime/docker/buildkit_trace_test.go cover: start/done/cached/error vertex transitions, log-line splitting, malformed input (silently ignored), unknown-field forward-compat
  • New integration test TestBuildKit_PerVertexEventsEmitted asserts ≥2 BuildLayerEvent + a BuildLogEvent containing a per-run unique RUN marker (cache-busts so the RUN vertex actually executes)
  • Mutation-tested the integration test: stubbed out the aux-routing line and reran — test failed with "got 0 (aux decoder dead?)", confirming the assertion catches a dead decoder

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Improved BuildKit trace decoding for richer per-step build progress and clearer build output.
  • Tests

    • Added unit tests covering trace parsing, log splitting, and robust handling of malformed/unknown inputs.
    • Added integration test validating per-vertex build events and log output during container builds.

Review Change Stack

streamBuildOutput now routes moby.buildkit.trace aux records to a
hand-rolled wire-format decoder (protowire) that parses the subset of
controlapi.StatusResponse we need: Vertex name/cached/started/completed/
error and VertexLog msg. Emits BuildLayerEvent on per-digest start and
complete transitions, BuildLogEvent per RUN log line.

Avoids pulling github.com/moby/buildkit as a direct dep (+253 transitive
modules: containerd, k8s, sigstore, AWS/Azure SDKs). Net cost is +2
modules (golang/protobuf v1 shim, xerrors) from promoting
google.golang.org/protobuf to direct.

Integration test builds a 2-step Dockerfile with a per-run-unique RUN
marker and asserts >=2 BuildLayerEvents + a BuildLogEvent containing the
marker, so a future regression in the aux routing goes loud rather than
silent.

Closes part of #50 (limitation 1). Limitations 2 (extractBaseImages
parser blind spots) and 3 (tarDirectory / .dockerignore) remain open.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 13, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7019be3a-42ac-4a8e-81c2-c8c5fc52bf9c

📥 Commits

Reviewing files that changed from the base of the PR and between 91a0373 and 25be89c.

📒 Files selected for processing (1)
  • runtime/docker/buildkit_trace.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • runtime/docker/buildkit_trace.go

📝 Walkthrough

Walkthrough

Adds a BuildKit trace decoder to parse moby.buildkit.trace aux protobuf payloads, integrates it into streamBuildOutput, and provides unit and integration tests to emit per-vertex layer and log events during Docker builds.

Changes

BuildKit Trace Event Decoding

Layer / File(s) Summary
Protobuf dependency
go.mod
Added google.golang.org/protobuf v1.34.2 to enable protobuf wire decoding.
BuildKit trace decoder implementation
runtime/docker/buildkit_trace.go
Implements buildkitTraceDecoder with wire-format parsing, per-digest deduplication, and emission of BuildEventLayer (START/DONE/ERROR/CACHED, once per digest) and BuildEventLog (per log line); malformed records are silently ignored.
streamBuildOutput integration
runtime/docker/build.go
Extended message parsing to include BuildKit id and aux fields; initializes decoder and routes moby.buildkit.trace messages with non-empty aux to trace.handleAux for event emission.
Unit tests for decoder
runtime/docker/buildkit_trace_test.go
Tests verify exact-once start/done/cached/error events per vertex, log line splitting, silent malformed input handling, and forward-compatibility with unknown protobuf fields; includes protobuf wire construction helpers.
Integration test
test/integration/buildkit_trace_events_test.go
Verifies a BuildKit build emits Dockerfile layer events and captures a unique RUN step marker in the log stream via buffered event collection and timeout-guarded execution.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • crunchloop/devcontainer#47: Both PRs modify runtime/docker/build.go's BuildKit progress handling; this PR adds aux trace decoding and per-vertex event emission.

Poem

🐰 I nibble wires and bytes with cheer,

Decoding traces, layer events appear,
Logs split like carrots, one per line,
Each vertex finished—hops, hop—it's fine,
A rabbit's clap for builds made clear.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 31.25% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding BuildKit aux trace record decoding to emit per-vertex events in the runtime module.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch buildkit-trace-decoder

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@runtime/docker/buildkit_trace.go`:
- Around line 103-108: The file has formatting drift causing lint failures; run
gofmt (or gofmt -w) on the file containing the buildkitTraceDecoder.decodeVertex
function to normalize indentation and spacing (e.g., align the var block and
surrounding code) so the decodedVertex method matches gofmt style; commit the
reformatted file.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e74b1b25-6154-4c88-8f52-9502bc1cb6f0

📥 Commits

Reviewing files that changed from the base of the PR and between d0fe747 and 91a0373.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (5)
  • go.mod
  • runtime/docker/build.go
  • runtime/docker/buildkit_trace.go
  • runtime/docker/buildkit_trace_test.go
  • test/integration/buildkit_trace_events_test.go

Comment thread runtime/docker/buildkit_trace.go
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bilby91 bilby91 merged commit 6db715d into main May 13, 2026
5 checks passed
@bilby91 bilby91 deleted the buildkit-trace-decoder branch May 14, 2026 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant