Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic when re-bootstrapping driver in non Running state #2464

Closed
3 tasks done
sgarcez opened this issue May 20, 2024 · 0 comments · Fixed by #2467
Closed
3 tasks done

Panic when re-bootstrapping driver in non Running state #2464

sgarcez opened this issue May 20, 2024 · 0 comments · Fixed by #2467
Labels
kind/bug Something isn't working status/triage

Comments

@sgarcez
Copy link
Member

sgarcez commented May 20, 2024

Contributing guidelines

I've found a bug and checked that ...

  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem

Description

Under very specific conditions bootstrapping a driver can panic here due to a nil progress.Writer.

The flow seems to be:

  • In build cmd drivers are resolved explicitly here via nodeResolver.Resolve
  • This calls nodeResolver.boot here with a valid progress.Writer
  • Later there is another boot call here via resolvedNode.Client
  • Which calls nodeResolver.boot here with a nil progress.Writer
  • The actual problem is that later on, here, driver.Boot calls driver.Info and, if the status is not Running, it tries to bootstrap it again.
  • This panics due to the nil progress.Writer

In order to reproduce this the driver must not be Running by the time the second boot call happens, which is a bit of an edge case, but possible AFAICT.
In order to force this to reproduce we can simply patch e.g. the remote driver to always return Inactive.

func (d *Driver) Info(ctx context.Context) (*driver.Info, error) {
	return &driver.Info{
		Status: driver.Inactive,
	}, nil
}

Expected behaviour

No panic. An error message related to the driver state not being Running.

Actual behaviour

◊ ./bin/build/buildx create --name remote --driver remote tcp://localhost:1234
◊ ./bin/build/buildx build --builder remote --progress plain .
#0 building with "remote" instance using remote driver

#1 [internal] waiting for connection
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x1017dce98]

goroutine 154 [running]:
github.com/docker/buildx/driver.Boot({0x102d4b880, 0x140003fee10}, {0x102d4b880, 0x140003fe5a0}, 0x14000581f90, {0x0?, 0x0})
        github.com/docker/buildx/driver/driver.go:96 +0x98
github.com/docker/buildx/build.(*nodeResolver).boot.func1.1({0x102d4b880?, 0x140003fee10?})
        github.com/docker/buildx/build/driver.go:244 +0x5c
github.com/moby/buildkit/util/flightcontrol.(*call[...]).run(0x102d58000)
        github.com/moby/buildkit@v0.13.0-rc3.0.20240417151852-71f99c52a669/util/flightcontrol/flightcontrol.go:121 +0x1e8
sync.(*Once).doSlow(0x0?, 0x100b95a54?)
        sync/once.go:74 +0x100
sync.(*Once).Do(0x1400061a0f0?, 0x140004d5200?)
        sync/once.go:65 +0x24
created by github.com/moby/buildkit/util/flightcontrol.(*call[...]).wait in goroutine 150
        github.com/moby/buildkit@v0.13.0-rc3.0.20240417151852-71f99c52a669/util/flightcontrol/flightcontrol.go:165 +0x504

Buildx version

github.com/docker/buildx v0.14.0-36-gafcb6099.m afcb609.m

Docker info

No response

Builders list

NAME/NODE                          DRIVER/ENDPOINT                                      STATUS     BUILDKIT   PLATFORMS
remote                             remote
 \_ remote0                         \_ tcp://localhost:1234                             running    v0.13.2    linux/arm64
default                            docker
 \_ default                         \_ default                                          running    v0.13.1    linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
desktop-linux*                     docker
 \_ desktop-linux                   \_ desktop-linux                                    running    v0.13.1    linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

Configuration

buildx checkout

◊ ./bin/build/buildx create --name remote --driver remote tcp://localhost:1234
◊ ./bin/build/buildx build --builder remote --progress plain .

Build logs

No response

Additional info

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working status/triage
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants