Skip to content

build: add --validate=repro reproducibility check#3872

Closed
lohitkolluri wants to merge 1 commit into
docker:masterfrom
lohitkolluri:lk/validate-repro
Closed

build: add --validate=repro reproducibility check#3872
lohitkolluri wants to merge 1 commit into
docker:masterfrom
lohitkolluri:lk/validate-repro

Conversation

@lohitkolluri
Copy link
Copy Markdown

Summary

  • Add an extensible --validate flag and implement --validate=repro.
  • Repro validation runs the build twice and fails if the resulting image digest differs.
  • The second run forces --no-cache.
  • If SOURCE_DATE_EPOCH is not explicitly set, it is pinned for both runs to make the digest comparison meaningful.

Test plan

  • go test ./commands/...
  • go build ./cmd/buildx

Closes #2053

Comment thread commands/build.go
Comment thread commands/build.go
@lohitkolluri lohitkolluri force-pushed the lk/validate-repro branch 2 times, most recently from fbb85f6 to 4d56471 Compare May 28, 2026 06:02
@lohitkolluri lohitkolluri requested a review from AkihiroSuda May 28, 2026 06:03
Comment thread commands/build.go Outdated
@lohitkolluri
Copy link
Copy Markdown
Author

I dug into the CI failures and they were mostly a single root cause fanning out across the matrix, plus one integration panic and docs drift.

  • Fixed lint-gopls by switching the BuildArgs clone to maps.Copy (the modernize analyzer was failing the job).
  • Fixed the integration failure by preventing a send on closed channel panic in the progress Printer during shutdown (writes are now best-effort when closing).
  • Updated the generated docs to include the new --validate flag.
  • Made validate-docs more deterministic by comparing the generated output to the checked-in docs directly (so failures show a real diff instead of a brittle git-status signal).
  • Normalized one path expectation in bake/entitlements_test.go for macOS /private path canonicalization.

Locally verified: make lint, make lint-gopls, make validate-docs, and go test ./... -short.

@lohitkolluri lohitkolluri requested a review from AkihiroSuda May 28, 2026 07:05
@AkihiroSuda
Copy link
Copy Markdown
Collaborator

I dug into the CI failures and they were mostly a single root cause fanning out across the matrix, plus one integration panic and docs drift.

Thanks, but please submit separate PRs

@lohitkolluri
Copy link
Copy Markdown
Author

Per review feedback, I split the follow-up CI fixes into separate PRs and kept this PR focused on --validate=repro (+ docs + the small gopls modernize fix).

Split PRs:

@lohitkolluri lohitkolluri force-pushed the lk/validate-repro branch 3 times, most recently from db34a07 to 98acfd7 Compare May 28, 2026 07:38
@lohitkolluri
Copy link
Copy Markdown
Author

@AkihiroSuda sorry for the trouble. Could you please approve the CI when you get a chance?

@lohitkolluri
Copy link
Copy Markdown
Author

Looks like the lint checks are failing across all platforms. I’ll patch the issues and push a follow-up commit shortly.

Fix gofmt import ordering so golangci-lint passes across the validate (lint, *) matrix.

Signed-off-by: Lohit Kolluri <lohitkolluri@gmail.com>
@lohitkolluri
Copy link
Copy Markdown
Author

lohitkolluri commented May 28, 2026

@AkihiroSuda I tracked down the CI failures, it turned out one file just needed a quick formatting cleanup. I fixed that and pushed an updated (squashed) commit to this PR. CI should be Green now.

@lohitkolluri
Copy link
Copy Markdown
Author

@AkihiroSuda looks like CI is passing on all 3 PR's

@crazy-max
Copy link
Copy Markdown
Member

Thanks for working on this, but we need to pause this PR for now.

The proposal in #2053 is still being discussed afaik. We have not yet aligned on whether this belongs in the Buildx CLI, whether it should live elsewhere such as the GitHub Action, or what the final flag/API shape should be.

Per our contributing guidelines, this kind of large or high-impact changes should be coordinated with maintainers before submitting an implementation PR. Adding a new buildx build flag and reproducibility validation behavior falls into that category.

Please continue the design discussion in #2053 and wait for maintainer agreement on the scope and UX before opening or continuing implementation PRs. Otherwise we spend maintainer review and CI time on code that may not be accepted.

I'm going to close this PR for now. Once the proposal is approved and the expected behavior is clearly defined, a focused implementation PR would be welcome.

@crazy-max crazy-max closed this May 28, 2026
@lohitkolluri
Copy link
Copy Markdown
Author

Understood that this needs maintainer alignment first. I’ll stop work on this PR and continue the design discussion in issue #2053 . Once there’s agreement on the expected behavior, I’m happy to come back with a smaller, focused implementation PR that matches it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add --twice (or --ntimes=N) flag to test reproducibility

4 participants