unknown service moby.sshforward.v1.SSH when building an image from github using buildkit

[DamianPereira]

Description

It seems related to moby/buildkit#2040, it was supposedly fixed by #3050, but it keeps happening in my Docker for mac 4.17.0 install.

Reproduce

DOCKER_BUILDKIT=1 docker build git@github.com:dockersamples/linux_tweet_app.git

This fails with the error: failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to load cache key: rpc error: code = Unimplemented desc = unknown service moby.sshforward.v1.SSH

but this succeeds:

DOCKER_BUILDKIT=0 docker build git@github.com:dockersamples/linux_tweet_app.git

Expected behavior

I expect build to pull the dockerfile from the github repository, read it and build it.

docker version

Client:
 Cloud integration: v1.0.31
 Version:           20.10.23
 API version:       1.41
 Go version:        go1.18.10
 Git commit:        7155243
 Built:             Thu Jan 19 17:35:19 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux
 Experimental:      true

Server: Docker Desktop 4.17.0 (99724)
 Engine:
  Version:          20.10.23
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.10
  Git commit:       6051f14
  Built:            Thu Jan 19 17:31:28 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.18
  GitCommit:        2456e983eb9e37e47538f59ea18f2043c9a73640
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.10.3)
  compose: Docker Compose (Docker Inc., v2.15.1)
  dev: Docker Dev Environments (Docker Inc., v0.1.0)
  extension: Manages Docker extensions (Docker Inc., v0.2.18)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.25.0)
  scout: Command line tool for Docker Scout (Docker Inc., v0.6.0)

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 33
 Server Version: 20.10.23
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.15.49-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 11.68GiB
 Name: docker-desktop
 ID: 6BJL:CA3W:QZ4J:GAGE:KXA5:S2PN:PDMH:YHVO:OYPQ:K537:G65I:2QQ2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

I tried restoring to factory defaults, and cloning many different images, but I just can't make it work with DOCKER_BUILDKIT=1 docker build.

Using command DOCKER_BUILDKIT=1 docker buildx build git@github.com:dockersamples/linux_tweet_app.git works and builds the image correctly, but I can not see it in the docker for mac images dashboard (it does show up in docker image ls)

[sam-thibault]

/cc @crazy-max is this expected? not yet supported in 20.10? With buildx build it works so maybe this is an issue in the CLI in 20.10.

[crazy-max]

Hum yes this might be an issue with CLI 20.10 and indeed looks similar to moby/buildkit#2040.

@DamianPereira Can you try with:

DOCKER_BUILDKIT=1 docker build --ssh default git@github.com:dockersamples/linux_tweet_app.git

[DamianPereira]

That works perfectly! It also work without DOCKER_BUILDKIT=1, which I think is the same thing since it's the default. Should it be working without --ssh default?

[DamianPereira]

Also, I know this might not be relevant to this repo, but do you know if docker compose up supports the --ssh default somehow? I can't seem to make it work (my original use case is a docker compose with build: git repo in it. I can make it work with docker compose build --ssh default followed by the up command.

docker/compose#9338 seems to suggest it should work, but my up command does not support --ssh.

[crazy-max]

Ok so #3050 has not been backported to the 20.10 branch:

cli/cli/command/image/build_buildkit.go

Lines 188 to 194 in 3825ed5

	if len(options.ssh) > 0 {
	sshp, err := parseSSHSpecs(options.ssh)
	if err != nil {
	return errors.Wrapf(err, "could not parse ssh: %v", options.ssh)
	}
	s.Allow(sshp)
	}

@thaJeztah Maybe we should?

[thaJeztah]

Yeah. It's not sure yet if there will be more 20.10 patch releases (and it probably is no issue with 23.0, right?). Guess it won't hurt to have a backport prepared at least; do you want to open one?

[crazy-max]

and it probably is no issue with 23.0, right?

Yes it's working with Docker 23 as Buildx is the default builder when BuildKit enabled (since Buildx 0.6.0: docker/buildx#581).

Guess it won't hurt to have a backport prepared at least; do you want to open one?

Sure

[DamianPereira]

It's weird, this command DOCKER_BUILDKIT=1 docker build --ssh default git@github.com:dockersamples/linux_tweet_app.git worked for me a few days ago, but it does not work today. Could something have changed with github certificates that might have caused this?

[humb1t]

Hey @DamianPereira, were you able to bypass this issue? I also want to use ssh links inside build.context and receive the error. Unfortunately, I can't use DOCKER_BUILDKIT=0.

[thaJeztah]

@humb1t what version of docker and buildx are you running? I think this should be resolved in current versions (the reported version (20.10) reached EOL, and the docker CLI itself now uses buildx under the hood to run docker build)