Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

failed to load LLB: runtime execution on platform linux/arm64 not supported #138

Closed
tamalsaha opened this issue Aug 22, 2019 · 17 comments
Closed

Comments

@tamalsaha
Copy link

@tamalsaha tamalsaha commented Aug 22, 2019

We started using the new docker buildx to build arm64 images. But it is not working with debian base image.

DOCKER_CLI_EXPERIMENTAL=enabled docker buildx build --platform linux/arm64 --load --pull -t kubedb/operator:v0.13.0-rc.0-dbg_linux_arm64 -f bin/.dockerfile .
container: kubedb/operator:v0.13.0-rc.0-dbg_linux_arm64
[+] Building 1.9s (3/3) FINISHED                                                                                                                                                                            
 => [internal] load build definition from .dockerfile-DBG-linux_arm64                                                                                                                                  0.5s
 => => transferring dockerfile: 652B                                                                                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                                                                                      0.3s
 => => transferring context: 2B                                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/library/debian:stretch                                                                                                                                      1.2s
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to load LLB: runtime execution on platform linux/arm64 not supported
Makefile:205: recipe for target 'bin/.container-kubedb_operator-v0.13.0-rc.0_linux_arm64-DBG' failed
make[2]: *** [bin/.container-kubedb_operator-v0.13.0-rc.0_linux_arm64-DBG] Error 1
Makefile:105: recipe for target 'push-linux_arm64' failed
make[1]: *** [push-linux_arm64] Error 2
Makefile:307: recipe for target 'release' failed
make: *** [release] Error 2

Here is the docker file:

FROM debian:stretch

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true

RUN set -x \
  && apt-get update \
  && apt-get install -y --no-install-recommends apt-transport-https ca-certificates tzdata locales openssl \
  && rm -rf /var/lib/apt/lists/* /usr/share/doc /usr/share/man /tmp/* \
  && localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8 \
  && echo 'Etc/UTC' > /etc/timezone && dpkg-reconfigure tzdata

ENV TZ     :/etc/localtime
ENV LANG   en_US.utf8
ENV LC_ALL en_US.UTF-8

ADD bin/linux_arm64/operator /operator

ENTRYPOINT ["/operator"]

I am getting anything with google search for failed to solve with frontend dockerfile.v0: failed to load LLB: runtime execution on platform linux/arm64 not supported.

On the same machine, the following arm64 image is building ok.

FROM gcr.io/distroless/base

ADD bin/linux_arm64/operator /operator

# This would be nicer as `nobody:nobody` but distroless has no such entries.
USER 65535:65535

ENTRYPOINT ["/operator"]

Any help is appreciated.

@tamalsaha

This comment has been minimized.

Copy link
Author

@tamalsaha tamalsaha commented Aug 22, 2019

I am running this on Ubuntu 18.04 and following https://community.arm.com/developer/tools-software/tools/b/tools-software-ides-blog/posts/getting-started-with-docker-for-arm-on-linux

$ uname -a
Linux beast 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

$ docker info
Client:
 Debug Mode: false
 Plugins:
  app: Docker Application (Docker Inc., v0.8.0)
  buildx: Build with BuildKit (Docker Inc., v0.2.2-10-g3f18b65-tp-docker)

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 105
 Server Version: 19.03.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.15.0-58-generic
 Operating System: Ubuntu 18.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.38GiB
 Name: beast
 ID: H747:N42V:6KTW:VQWA:RUJL:6U73:XTFK:VSPF:GWTL:KHMA:POO7:O2AN
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: tigerworks
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support

@tonistiigi

This comment has been minimized.

Copy link
Member

@tonistiigi tonistiigi commented Aug 22, 2019

you need to install qemu-user #132 (comment)

@tamalsaha

This comment has been minimized.

Copy link
Author

@tamalsaha tamalsaha commented Aug 22, 2019

Thanks @tonistiigi . It seems that I had already installed it but still getting this error.

$ sudo apt-get install qemu-user-static
[sudo] password for tamal: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
qemu-user-static is already the newest version (1:2.11+dfsg-1ubuntu7.17).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

$ env DOCKER_CLI_EXPERIMENTAL=enabled docker buildx build --platform linux/arm64 --load --pull -t tigerworks/operator:fdocker .
[+] Building 1.5s (3/3) FINISHED                                                                                                                                                                            
 => [internal] load .dockerignore                                                                                                                                                                      0.4s
 => => transferring context: 2B                                                                                                                                                                        0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                   0.6s
 => => transferring dockerfile: 32B                                                                                                                                                                    0.0s
 => [internal] load metadata for docker.io/library/debian:stretch                                                                                                                                      0.7s
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to load LLB: runtime execution on platform linux/arm64 not supported
@tamalsaha

This comment has been minimized.

Copy link
Author

@tamalsaha tamalsaha commented Aug 22, 2019

I had a mybuilder that was not working.

$ env DOCKER_CLI_EXPERIMENTAL=enabled docker buildx ls
NAME/NODE    DRIVER/ENDPOINT             STATUS  PLATFORMS
mybuilder *  docker-container                    
  mybuilder0 unix:///var/run/docker.sock running linux/amd64, linux/386
default      docker                              
  default    default                     running linux/amd64

I rm-rf the old builder and then created a new one and things are working now.

$ docker buildx create --name nubuilder
nubuilder
$ docker buildx use nubuilder
$ docker buildx inspect --bootstrap
[+] Building 9.6s (1/1) FINISHED                                                                                                                                                                            
 => [internal] booting buildkit                                                                                                                                                                        9.6s
 => => pulling image moby/buildkit:master                                                                                                                                                              5.8s
 => => creating container buildx_buildkit_nubuilder0                                                                                                                                                   3.8s
Name:   nubuilder
Driver: docker-container

Nodes:
Name:      nubuilder0
Endpoint:  unix:///var/run/docker.sock
Status:    running
Platforms: linux/amd64, linux/arm64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

So, the problem was the old builder did not have arm64. I don't know why. Thanks @carlosedp helping me get this working.

@tamalsaha tamalsaha closed this Aug 22, 2019
@tonistiigi

This comment has been minimized.

Copy link
Member

@tonistiigi tonistiigi commented Aug 23, 2019

You need to restart or recreate the builder for it to pick up new qemu support.

@evandronas

This comment has been minimized.

Copy link

@evandronas evandronas commented Aug 30, 2019

Hi, have the same problem, am working with rhel7 and runed'install qemu-user', but when run 'buildx ls', it show only 'running linux/amd64, linux/386', and it don't show 'linux/amd64, linux/arm64...'. What can I have do, please?

@tamalsaha

This comment has been minimized.

Copy link
Author

@tamalsaha tamalsaha commented Aug 30, 2019

@evandronas

This comment has been minimized.

Copy link

@evandronas evandronas commented Aug 30, 2019

Sorry, i am begin work with this now, how can I delete the old builder e create the new?

@tamalsaha

This comment has been minimized.

Copy link
Author

@tamalsaha tamalsaha commented Aug 30, 2019

@evandronas

This comment has been minimized.

Copy link

@evandronas evandronas commented Aug 30, 2019

ok, thanks a lot! I will do this.

@evandronas

This comment has been minimized.

Copy link

@evandronas evandronas commented Aug 30, 2019

I see I need to read a little more about it, but I'm having a little trouble finding something about using this buildx technology ... I'd like to create a raspbian / arm docker image in my rhel7 environment, but I'm with some difficulty. Could you please pass some link with material that could help me. I'll be very grateful. Thank you!

@tamalsaha

This comment has been minimized.

Copy link
Author

@tamalsaha tamalsaha commented Aug 30, 2019

@tukno

This comment has been minimized.

Copy link

@tukno tukno commented Sep 20, 2019

It worked for me after installing qemu-user and running the getting started tutorial at multiarch/qemu-user-static

@wohali

This comment has been minimized.

Copy link

@wohali wohali commented Dec 6, 2019

I found that on my work image, I have to constantly re-run the multiarch/qemu-user-static process after each boot. It's a simple enough workaround for us anyway.

@hfedcba

This comment has been minimized.

Copy link

@hfedcba hfedcba commented Dec 27, 2019

For me running

docker run --rm --privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d

solved the problem.

@HeroCC

This comment has been minimized.

Copy link

@HeroCC HeroCC commented Jan 7, 2020

Sorry to bug anyone, but I'm having the same issue. I've been trying to get this to work for a while now but can't seem to do it. I run the following, and yet my buildx builder doesn't have the new platforms supported, and the build gets that same error. The base image I'm trying to build is ubuntu:18.04 and I'm running on GitlabCI shared infrastructure (docker:stable image, dind service running, privileged containers enabled).

docker run --rm --privileged multiarch/qemu-user-static:register --reset
docker buildx create --use
docker buildx inspect --bootstrap
  * Platforms: linux/amd64, linux/386
docker buildx build --load --platform "linux/amd64,linux/arm64,linux/arm/v7" -t name .

Full Build Logs: https://gitlab.com/moos-ivp/moosdocker/-/jobs/396347211
CI Configuration / scripts: https://gitlab.com/moos-ivp/moosdocker/blob/d8bc10615ddf928c8fda8cf250cfa2c973734e29/.gitlab-ci.yml

Thank you!

@Silex

This comment has been minimized.

Copy link

@Silex Silex commented Jan 10, 2020

@HeroCC: try docker run --rm --privileged multiarch/qemu-user-static --reset -p yes instead and it should work

Note that docker buildx ls will not list linux/arm/v7 as platform but it works.

HeroCC added a commit to HeroCC/MOOSDocker that referenced this issue Jan 14, 2020
As suggested by someone on the QEmu-user-static Github page: docker/buildx#138 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants
You can’t perform that action at this time.