vendor: github.com/moby/buildkit v0.13.0-rc2 #47364

vvoland · 2024-02-09T14:38:31Z

- What I did
Updated buildkit to v0.13.0-rc2

- How I did it
See individual commits.

- How to verify it

- Description for the changelog

- Update buildkit to v0.13.0-rc2

- A picture of a cute animal (not mandatory but encouraged)

thaJeztah · 2024-02-09T15:18:33Z

@stewartsmith @PeterCai7 are you able to help with the cloudwatch logger?

@vvoland perhaps we can move that update to a separate PR in preparation?

vendor.mod

crazy-max · 2024-02-09T17:42:06Z

Thanks for looking at it @vvoland, I will review your comments shortly

vvoland · 2024-02-12T17:32:51Z

Extracted the AWS bump to: #47373

builder/builder-next/exporter/mobyexporter/export.go

vvoland · 2024-02-14T12:12:08Z

Looks like increasing the integration tests timeout from 5m to 10m solves the TestWaitConditions failure.
Wondering what's caused the tests to execute longer though 🤔

Comparing (this PR):
https://github.com/moby/moby/actions/runs/7889967599#summary-21563428016
vs (latest master)
https://github.com/moby/moby/actions/runs/7886719263#summary-21521123926

looks like indeed the tests take more time to complete, for example:
TestServicePlugin from ~28s to ~67s
TestLiveRestore/volume_references from ~17s to ~88s

these don't use build related things (and integration tests will use the legacy builder anyway), so wondering if it's the OTEL stuff that's making things slower?

EDIT: Solved with d32b3cf (OTEL was trying to export spans over a wrong protocol and was slowing down all scopes that were being traced).

vvoland · 2024-02-15T14:28:42Z

This also fails consistently:

=== Failed
=== FAIL: amd64.integration.daemon TestDaemonProxy/environment_variables (11.81s)
    daemon_test.go:229: assertion failed:  (received string) != example.org:5000 (string): should not have used proxy
    --- FAIL: TestDaemonProxy/environment_variables (11.81s)

=== FAIL: amd64.integration.daemon TestDaemonProxy/configuration_file (11.81s)
    daemon_test.go:330: assertion failed:  (received string) != example.org:5002 (string): should not have used proxy
    --- FAIL: TestDaemonProxy/configuration_file (11.81s)

=== FAIL: amd64.integration.daemon TestDaemonProxy/command-line_options (11.82s)
    daemon_test.go:280: assertion failed:  (received string) != example.org:5001 (string): should not have used proxy
    --- FAIL: TestDaemonProxy/command-line_options (11.82s)

=== FAIL: amd64.integration.daemon TestDaemonProxy (0.50s)

vvoland · 2024-02-23T18:31:20Z

Rebased after #47245 merge

vvoland · 2024-02-26T10:17:06Z

Updated to rc2

builder/builder-next/adapters/containerimage/pull.go

slonopotamus · 2024-02-27T09:12:16Z

builder/builder-next/adapters/containerimage/pull.go

 			if err != nil {
 				return struct{}{}, err
 			}

-			p.ref = newRef


Is this correct?

Oh, good catch!

Yeah it looks like this shouldn't be deleted.
I restored it as:
p.ref = ref.String()

because ResolveImageConfig no longer returns a reference.
Looking at the old implementation, it didn't really change the ref that was passed as an argument.
The only place where it returned a different ref is where it went to the (remotes.Resolver) Resolve(), but it doesn't seem to change it either (at least the docker remote).

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com> Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Introduced in: moby/buildkit@3713178 Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

StaticDirSource definition changed and can no longer be initialized from the composite literal. https://github.com/moby/buildkit/commits/a80b48544c029f0e28e1243407b55c916c7757f2 Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

moby/buildkit@1c1777b added an explicit id argument to the Resolve method. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

moby/buildkit@e358792 changed that field to a function and added an `OverrideResource` function that allows to override it. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

moby/buildkit@8bfd280 added a new argument that allows to specify different runtime. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Adjust to cache sources changes from: moby/buildkit@6b27487 Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

moby/buildkit@30c069c removed the `ResolveImageConfig` method in favor of more generic `ResolveSourceMetadata` that can also support other things than images. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Temporarily store the produced config descriptor for the buildkit history to work. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com> Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Buildkit added support for exporting metrics in: moby/buildkit@7de2e4f Explicitly set the protocol for exporting metrics like we do for the traces. We need that because Buildkit defaults to grpc. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

thaJeztah

LGTM

discussed on Slack; looks like the failures are flaky tests, so no need to re-run CI again for now.

vvoland added area/builder status/2-code-review kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. area/builder/buildkit Issues affecting buildkit containerd-integration Issues and PRs related to containerd integration labels Feb 9, 2024

vvoland added this to the 26.0.0 milestone Feb 9, 2024

vvoland self-assigned this Feb 9, 2024

This comment was marked as off-topic.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as off-topic.

Sign in to view

vvoland force-pushed the buildkit-v13 branch from 80b30c9 to ec937c9 Compare February 9, 2024 15:37

This comment was marked as off-topic.

Sign in to view

crazy-max reviewed Feb 9, 2024

View reviewed changes

vendor.mod Outdated Show resolved Hide resolved

thaJeztah mentioned this pull request Feb 12, 2024

vendor: github.com/containerd/nydus-snapshotter v0.13.7 #47371

Merged

vvoland force-pushed the buildkit-v13 branch from ec937c9 to 18b2ec4 Compare February 12, 2024 17:13

vvoland mentioned this pull request Feb 12, 2024

vendor: bump github.com/aws/aws-sdk-go-v2 to v1.24.1 #47373

Merged

vvoland force-pushed the buildkit-v13 branch from 18b2ec4 to de9995f Compare February 12, 2024 17:28

vvoland force-pushed the buildkit-v13 branch from de9995f to c02ca9f Compare February 13, 2024 15:54

vvoland commented Feb 13, 2024

View reviewed changes

builder/builder-next/exporter/mobyexporter/export.go Show resolved Hide resolved

vvoland force-pushed the buildkit-v13 branch from 0fcd7b9 to 5dc8d1d Compare February 15, 2024 11:04

crazy-max mentioned this pull request Feb 15, 2024

client: fix multiple exporter test for dockerd worker moby/buildkit#4654

Merged

vvoland force-pushed the buildkit-v13 branch 2 times, most recently from 0bda842 to 1638943 Compare February 16, 2024 10:44

vvoland mentioned this pull request Feb 22, 2024

[DNM] Buildkit v0.13 + OTEL v1.21 #47435

Closed

vvoland added the area/dependencies label Feb 23, 2024

vvoland force-pushed the buildkit-v13 branch from e4a6f85 to e6dfbdc Compare February 23, 2024 18:30

cpuguy83 approved these changes Feb 23, 2024

View reviewed changes

vvoland force-pushed the buildkit-v13 branch from e6dfbdc to b84098f Compare February 26, 2024 10:16

vvoland changed the title ~~vendor: github.com/moby/buildkit v0.13.0-rc1~~ vendor: github.com/moby/buildkit v0.13.0-rc2 Feb 26, 2024

slonopotamus reviewed Feb 26, 2024

View reviewed changes

builder/builder-next/adapters/containerimage/pull.go Show resolved Hide resolved

crazy-max approved these changes Feb 27, 2024

View reviewed changes

vvoland force-pushed the buildkit-v13 branch from b84098f to 33143e3 Compare February 27, 2024 09:05

slonopotamus reviewed Feb 27, 2024

View reviewed changes

crazy-max and others added 14 commits February 27, 2024 11:25

go.mod: bump to go 1.21 and use local toolchain when vendoring

f90b03e

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com> Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

vendor: github.com/moby/buildkit v0.13.0-rc2

31545c3

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

integration/build: Use new buildkit progressui

dd69926

Introduced in: moby/buildkit@3713178 Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

integration/build: Use fsutil.NewFS

e68f712

StaticDirSource definition changed and can no longer be initialized from the composite literal. https://github.com/moby/buildkit/commits/a80b48544c029f0e28e1243407b55c916c7757f2 Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

builder-next/exporter: Sync with new signature

a79bb1e

moby/buildkit@1c1777b added an explicit id argument to the Resolve method. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

cmd/dockerd: Fix overriding OTEL resource

bc6d88c

moby/buildkit@e358792 changed that field to a function and added an `OverrideResource` function that allows to override it. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

builder/controller: Adjust NewWorkerOpt call

b04a2da

moby/buildkit@8bfd280 added a new argument that allows to specify different runtime. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

builder-next: Adjust to source changes

59ad169

Adjust to cache sources changes from: moby/buildkit@6b27487 Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

builder-next/mobyexporter: Use OptKeyName const

fa467ca

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

builder/mobyexporter: Set image.name response key

e01a1c5

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

mobyexporter: Store temporary config descriptor

f5722da

Temporarily store the produced config descriptor for the buildkit history to work. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

ci(buildkit): dedicated step to build test image

60358bf

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com> Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

vvoland force-pushed the buildkit-v13 branch from 33143e3 to 2d31532 Compare February 27, 2024 10:31

thaJeztah approved these changes Feb 27, 2024

View reviewed changes

thaJeztah merged commit 2208351 into moby:master Feb 27, 2024
134 of 138 checks passed

thaJeztah added the impact/changelog label Feb 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vendor: github.com/moby/buildkit v0.13.0-rc2 #47364

vendor: github.com/moby/buildkit v0.13.0-rc2 #47364

vvoland commented Feb 9, 2024 •

edited

This comment was marked as off-topic.

This comment was marked as resolved.

This comment was marked as resolved.

thaJeztah commented Feb 9, 2024

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

crazy-max commented Feb 9, 2024

vvoland commented Feb 12, 2024

vvoland commented Feb 14, 2024 •

edited

vvoland commented Feb 15, 2024

vvoland commented Feb 23, 2024

vvoland commented Feb 26, 2024

slonopotamus Feb 27, 2024

vvoland Feb 27, 2024

thaJeztah left a comment

vendor: github.com/moby/buildkit v0.13.0-rc2 #47364

vendor: github.com/moby/buildkit v0.13.0-rc2 #47364

Conversation

vvoland commented Feb 9, 2024 • edited

This comment was marked as off-topic.

This comment was marked as resolved.

This comment was marked as resolved.

thaJeztah commented Feb 9, 2024

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

crazy-max commented Feb 9, 2024

vvoland commented Feb 12, 2024

vvoland commented Feb 14, 2024 • edited

vvoland commented Feb 15, 2024

vvoland commented Feb 23, 2024

vvoland commented Feb 26, 2024

slonopotamus Feb 27, 2024

Choose a reason for hiding this comment

vvoland Feb 27, 2024

Choose a reason for hiding this comment

thaJeztah left a comment

Choose a reason for hiding this comment

vvoland commented Feb 9, 2024 •

edited

vvoland commented Feb 14, 2024 •

edited