fix(docker): worker build broke — go mod tidy pruned all deps pre-source#655
Merged
Conversation
The prewarm restructure in #654 moved the per-DuckDB-version pin into a module-files-only stage (COPY go.mod go.sum; go get; go mod tidy) ahead of COPY . .. With no .go files present, `go mod tidy` sees zero imports and prunes EVERY require directive, emptying go.mod. The stashed go.mod.pinned was therefore empty too, and restoring it after COPY . . gave the final build a go.mod with no dependencies: duckdbservice/appender_init.go:7:2: no required module provides package github.com/duckdb/duckdb-go/v2 All four worker matrix builds failed on main (container-image-worker-cd). Main + control-plane images were unaffected — they never run tidy. Fix: pin with `go get pkg@ver` (which records the require regardless of imports) but do NOT tidy in the source-less stage; drop the stash/restore. Re-run the pin + `go mod tidy` AFTER COPY . ., once the real import set is visible, so tidy keeps every needed require. The prewarm + module caches from the pre-COPY layers stay valid (same versions → go get is a fast metadata no-op). Verified locally: `go mod tidy` with only go.mod/go.sum prunes duckdb-go to zero requires; `go get duckdb-go/v2@v2.10502.0` without tidy keeps and pins it. Default-path worker binary builds clean.
benben
added a commit
that referenced
this pull request
Jun 2, 2026
The prewarm added in #654 was premised on a multi-minute CGO compile of duckdb-go. That premise was wrong: duckdb-go-bindings ships PREBUILT static libs (libduckdb_*.a etc, ~1.7GB in the module cache) — there is no DuckDB C++ compile to cache. The other prewarmed dep, pg_query_go, cold-compiles in ~4s. So the prewarm layer saved single-digit seconds while adding a confusing extra build step. Measured on warm main builds after #654: main 288s -> 264s controlplane 241s -> 229s The ~10% saving came from the LAYER REORDERING (module + extension downloads moved ahead of COPY . ., so they cache on source-only PRs), not from the prewarm. The dominant remaining cost is the final CGO *link* of the prebuilt static libs (~68s), which caching can't touch because it changes with every first-party source edit. Remove the prewarm RUN from all three Dockerfiles. Keep the valuable reordering and the worker pin-without-tidy fix from #655. Net build-time wins still live here: - module download + extension fetch cached before COPY . . Real PR-latency lever (separate change): build only linux/arm64 in PR CI (mw-dev is arm64), halving the worker matrix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Main is currently red for the worker image — all four
container-image-worker-cdmatrix builds fail onc4db37d(the #654merge). Forward-fix.
Cause
#654 moved the per-DuckDB-version pin into a module-files-only stage
(
COPY go.mod go.sum;go get;go mod tidy) ahead ofCOPY . ..With no
.gofiles present,go mod tidysees zero imports andprunes every
requiredirective, emptyinggo.mod. The stashedgo.mod.pinnedwas empty too, so restoring it afterCOPY . .gave thefinal build a depless go.mod:
Main + control-plane images were unaffected — they never run
tidy.Fix
go get pkg@ver(records therequireregardless of imports)but do not
tidyin the source-less stage.go.mod.pinnedstash/restore.go mod tidyafterCOPY . ., once the realimport set is visible, so tidy keeps every needed
require.Prewarm + module caches from the pre-COPY layers stay valid (same
versions →
go getis a fast metadata no-op, prewarmed objects reused).Verified locally
Default-path worker binary builds clean.
🤖 Generated with Claude Code