[Question] Question on the cross platform image build

[chendave]

I saw some examples, "examples/buildkit0/buildkit.go" for example, need to specify the platform information, amd64, arm64, ppc64le etc. If the value defined in the source doesn't match with the build server, it will throw below error messages,

"runtime execution on platform xxx not supported ..."

I am wondering what's the connection between the image and the build server? shouldn't the image works for different platform?

Thanks in the advance!

[tonistiigi]

I am wondering what's the connection between the image and the build server? shouldn't the image works for different platform?

Not sure I understand the question. An image can be multi-platform if it is a manifest list and contains subimages for different platforms. When you run a process as part of your build your build node needs to be capable to execute binaries for a specific platform though and that is where you see this error.

The LLB protobuf is specific to a single platform so that we can cache it properly (eg. will not match cache for amd64 for arm). To build for multiple platforms you need send different variants of protobuf for multiple platforms (theoretically you can have different vertexes of llb graph use different platforms as well but that is less useful).

See:
https://asciinema.org/a/GYOx4B88r272HWrLTyFwo156s
#550 (comment)

[chendave]

Thank you for explanation! I saw your demo and tried to follow it, the "buildctl build" run successfully with the below output,

buildctl build --frontend dockerfile.v0 --local dockerfile=. --local context=. --exporter image --exporter-opt name=docker.io/tonistiigi/curl:latest

[+] Building 11.6s (6/6) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 89B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 4.5s
=> [1/2] FROM docker.io/library/alpine@sha256:5a32c65954238c8f70fb9652ba7af86dbd3a10c053258578efea3 2.5s
=> => resolve docker.io/library/alpine@sha256:5a32c65954238c8f70fb9652ba7af86dbd3a10c053258578efea3 0.0s
=> => sha256:5a32c65954238c8f70fb9652ba7af86dbd3a10c053258578efea39897c127bf4 433B / 433B 0.0s
=> => sha256:25b4d910f4b76a63a3b45d0f69a57c34157500faf6087236581eca221c62d214 528B / 528B 0.0s
=> => sha256:6c40cc604d8e4c121adcb6b0bfe8bb038815c350980090e74aa5a6423f8f82c0 2.75MB / 2.75MB 2.3s
=> => sha256:caf27325b298a6730837023a8a342699c8b7b388b8d878966b064a1320043019 1.51kB / 1.51kB 0.0s
=> => unpacking docker.io/library/alpine@sha256:5a32c65954238c8f70fb9652ba7af86dbd3a10c053258578efe 0.2s
=> [2/2] RUN apk add --no-cache curl 4.3s
=> exporting to image 0.3s
=> => exporting layers 0.2s
=> => exporting manifest sha256:66bba584562da4da074a2053dea17e1b5a5d66107494bb4466b6f79cb45071d8 0.0s
=> => exporting config sha256:911094989c4e1e3da0007dcfcfd11db4b6e5dfa9f7810f6cde2aa5b3d93105fc 0.0s
=> => naming to docker.io/tonistiigi/curl:latest

But "ctr image ls" show nothing, "docker image ls" or "runc list" doesn't give me anything about the image, so where is the image created? is there any log information somewhere I can take a look?

ctr image ls

REF TYPE DIGEST SIZE PLATFORMS LABELS

Thanks a lot!

[chendave]

ha, I saw it in this way:

ctr --namespace=buildkit images ls

REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/tonistiigi/curl:latest application/vnd.docker.distribution.manifest.v2+json sha256:66bba584562da4da074a2053dea17e1b5a5d66107494bb4466b6f79cb45071d8 3.4 MiB linux/amd64 -

:)

[chendave]

Looks like the build node I am using only support "amd64", but seen from the video your node supports multiple platforms, how the buildctl get such information? in which way the node I am using could support multiple platforms like yours.

$ buildctl debug workers
ID PLATFORMS
a00k7jqs9wh2al5vzwar4ffx2 linux/amd64

Huge thanks!

[tonistiigi]

see the second link I posted #550 (comment) above about how to make workers aware of multiple arch

[chendave]

@tonistiigi Do you have any idea on this? I am on ubuntu16.04 and have upgrade the kernel to "4.4.14-040414-generic #201606241434"

# echo ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm-static:OCF' > /proc/sys/fs/binfmt_misc/register -bash: echo: write error: Invalid argument

[chendave]

upgraded to the kernel 4.16.0-041600-generic, then issue fixed, thank you!