s390x: build fedora binaries for peerpod #1678

huoqifeng · 2024-01-26T04:46:09Z

As a 1st step to enable mkosi images on s390x on fedora, we'll enable s390x binaries build on fedora with same Dockerfile on a s390x host in this PR.

There are other options to build s390x binaries on fedora like:

Build the s390x binaries on x86 host and s390x container
Build the s390x binaries on x86 host and x86 container with compilers cross platform
But it looks not easy, as we tried, both of the other 2 options failed. We'll track and maybe switch the approach in further PR as investigation move ahead.

In this PR, the fedora Dockerfile is revised following same format as ubuntu. It'll be more easier to maintain the Dockerfile.

To try the changes:

on s390x
To run the builder build:

docker buildx build \
	-t fedora-binaries-builder-s390x \
	--build-arg ARCH=s390x \
	--build-arg PROTOC_ARCH=s390x_64 \
	--build-arg YQ_ARCH=s390x \
	--build-arg YQ_CHECKSUM=sha256:4e6324d08630e7df733894a11830412a43703682d65a76f1fc925aac08268a45 \
	--load \
	-f podvm/Dockerfile.podvm_builder.fedora .

To run the binaries build against the builder image

docker buildx build \
	--build-arg BUILDER_IMG=fedora-binaries-builder-s390x:latest \
	--build-arg AA_KBC=offline_fs_kbc \
	-o type=local,dest="./resources/binaries-tree" \
	-f podvm/Dockerfile.podvm_binaries.fedora .

on x86

docker buildx build \
	-t fedora-binaries-builder-amd64 \
	--load \
	-f podvm/Dockerfile.podvm_builder.fedora .

docker buildx build \
	--build-arg BUILDER_IMG=fedora-binaries-builder-amd64:latest \
	--build-arg AA_KBC=offline_fs_kbc \
	-o type=local,dest="./resources/binaries-tree" \
	-f podvm/Dockerfile.podvm_binaries.fedora .

Note: BUILDER_IMG might need be pushed to a registry before use it.

Fixes: confidential-containers#1640 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

huoqifeng · 2024-01-26T04:58:26Z

The Dockerfile and Makefile change was verified both on x86 and s390x host with commands like:

root@c37646v1:~/src/cloud-api-adaptor/podvm-mkosi# make fedora-binaries-builder
Building fedora-binaries-builder image...
docker buildx build \
	-t fedora-binaries-builder \
	--load \
	-f ../podvm/Dockerfile.podvm_builder.fedora ../.
[+] Building 10.0s (23/23) FINISHED                                                                                                                           docker-container:multi-arch-builder
 => [internal] load build definition from Dockerfile.podvm_builder.fedora                                                                                                                    0.0s
 => => transferring dockerfile: 1.95kB                                                                                                                                                       0.0s
 => resolve image config for docker.io/docker/dockerfile:1.5-labs                                                                                                                            0.7s
 => [auth] docker/dockerfile:pull token for registry-1.docker.io                                                                                                                             0.0s
 => CACHED docker-image://docker.io/docker/dockerfile:1.5-labs@sha256:f2e91734a84c0922ff47aa4098ab775f1dfa932430d2888dd5cad5251fafdac4                                                       0.0s
 => => resolve docker.io/docker/dockerfile:1.5-labs@sha256:f2e91734a84c0922ff47aa4098ab775f1dfa932430d2888dd5cad5251fafdac4                                                                  0.0s
 => [internal] load .dockerignore                                                                                                                                                            0.0s
 => => transferring context: 254B                                                                                                                                                            0.0s
 => [internal] load metadata for registry.fedoraproject.org/fedora:38                                                                                                                        0.3s
 => [ 1/11] FROM registry.fedoraproject.org/fedora:38@sha256:6349d2df6b4322c5690df1bb7743c45c356e20471dda69f27218cd9ba4a6c3c7                                                                0.0s
 => => resolve registry.fedoraproject.org/fedora:38@sha256:6349d2df6b4322c5690df1bb7743c45c356e20471dda69f27218cd9ba4a6c3c7                                                                  0.0s
 => https://github.com/mikefarah/yq/releases/download/v4.35.1/yq_linux_amd64                                                                                                                 0.0s
 => https://github.com/protocolbuffers/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip                                                                                     0.7s
 => https://sh.rustup.rs                                                                                                                                                                     0.2s
 => https://dl.google.com/go/go1.20.12.linux-amd64.tar.gz                                                                                                                                    0.2s
 => CACHED [ 2/11] RUN dnf groupinstall -y 'Development Tools' &&     dnf install -y yum-utils gnupg git perl-core pkg-config libseccomp-devel gpgme-devel         device-mapper-devel unzi  0.0s
 => CACHED [ 3/11] ADD https://dl.google.com/go/go1.20.12.linux-amd64.tar.gz go1.20.12.linux-amd64.tar.gz                                                                                    0.0s
 => CACHED [ 4/11] RUN rm -rf /usr/local/go && tar -C /usr/local -xzf go1.20.12.linux-amd64.tar.gz && rm -f go1.20.12.linux-amd64.tar.gz                                                     0.0s
 => CACHED [ 5/11] ADD --checksum=sha256:bd695a6513f1196aeda17b174a15e9c351843fb1cef5f9be0af170f2dd744f08 https://github.com/mikefarah/yq/releases/download/v4.35.1/yq_linux_amd64 /usr/loc  0.0s
 => CACHED [ 6/11] RUN chmod a+x /usr/local/bin/yq                                                                                                                                           0.0s
 => CACHED [ 7/11] ADD https://sh.rustup.rs rustup                                                                                                                                           0.0s
 => CACHED [ 8/11] RUN chmod a+x rustup && ./rustup -y --default-toolchain 1.72.0                                                                                                            0.0s
 => CACHED [ 9/11] ADD https://github.com/protocolbuffers/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip protoc-3.11.4-linux-x86_64.zip                                   0.0s
 => CACHED [10/11] RUN unzip protoc-3.11.4-linux-x86_64.zip -d /usr/local && rm -f protoc-3.11.4-linux-x86_64.zip                                                                            0.0s
 => CACHED [11/11] WORKDIR /src                                                                                                                                                              0.0s
 => exporting to docker image format                                                                                                                                                         7.7s
 => => exporting layers                                                                                                                                                                      0.0s
 => => exporting manifest sha256:8cd3d93170c00ea7fe67c6f40d58c61f472ac54f344989e71e7f6b8f176f6045                                                                                            0.0s
 => => exporting config sha256:e579212e14775913e8a6c69382aadaef0df93d3a42c8097cb7c3654f14e93a9c                                                                                              0.0s
 => => sending tarball                                                                                                                                                                       7.6s
 => importing to docker      

root@c37646v1:~/src/cloud-api-adaptor/podvm-mkosi# docker tag fedora-binaries-builder:latest ibmhuoqif/fedora-binaries-builder-amd64:latest
root@c37646v1:~/src/cloud-api-adaptor/podvm-mkosi# docker push ibmhuoqif/fedora-binaries-builder-amd64:latest

root@c37646v1:~/src/cloud-api-adaptor/podvm-mkosi# make binaries
docker buildx use default
Building binaries...
rm -rf ./resources/binaries-tree
docker buildx build \
	--build-arg BUILDER_IMG=ibmhuoqif/fedora-binaries-builder-amd64:latest \
	--build-arg AA_KBC=offline_fs_kbc \
	-o type=local,dest="./resources/binaries-tree" \
	-f ../podvm/Dockerfile.podvm_binaries.fedora ../.
[+] Building 911.0s (10/10) FINISHED                                                                                                                                               docker:default
 => [internal] load .dockerignore                                                                                                                                                            0.1s
 => => transferring context: 254B                                                                                                                                                            0.0s
 => [internal] load build definition from Dockerfile.podvm_binaries.fedora                                                                                                                   0.1s
 => => transferring dockerfile: 913B                                                                                                                                                         0.0s
 => [internal] load metadata for docker.io/ibmhuoqif/fedora-binaries-builder-amd64:latest                                                                                                    0.0s
 => [internal] load build context                                                                                                                                                            1.3s
 => => transferring context: 203.42MB                                                                                                                                                        1.2s
 => [podvm_builder 1/4] FROM docker.io/ibmhuoqif/fedora-binaries-builder-amd64:latest                                                                                                        0.4s
 => [podvm_builder 2/4] COPY . /src/cloud-api-adaptor                                                                                                                                        0.5s
 => [podvm_builder 3/4] WORKDIR /src/cloud-api-adaptor/podvm                                                                                                                                 0.1s
 => [podvm_builder 4/4] RUN LIBC=gnu make binaries                                                                                                                                         906.8s
 => [stage-1 1/1] COPY --from=podvm_builder /src/cloud-api-adaptor/podvm/files /                                                                                                             0.4s 
 => exporting to client directory                                                                                                                                                            0.5s 
 => => copying files 174.76MB

liudalibj · 2024-01-26T05:26:13Z

Thanks @huoqifeng
I built out the docker.io/library/fedora-binaries-builder image on my s390x vsi, the image is built out but some require packages are BAD. Is this a support user case: build and run the fedora-builder container on s390x host directly?

root@liudali-s390x-libvirt16:~/cloud-api-adaptor/podvm-mkosi# make fedora-binaries-builder
Building fedora-binaries-builder image...
docker buildx build \
	-t fedora-binaries-builder \
	--load \
	-f ../podvm/Dockerfile.podvm_builder.fedora ../.
[+] Building 225.1s (21/21) FINISHED
 => [internal] load build definition from Dockerfile.podvm_builder.fedora                                                                                                                                                                       0.0s
 => => transferring dockerfile: 1.95kB                                                                                                                                                                                                          0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                               0.0s
 => => transferring context: 254B                                                                                                                                                                                                               0.0s
 => resolve image config for docker.io/docker/dockerfile:1.5-labs                                                                                                                                                                               0.6s
 => docker-image://docker.io/docker/dockerfile:1.5-labs@sha256:f2e91734a84c0922ff47aa4098ab775f1dfa932430d2888dd5cad5251fafdac4                                                                                                                 0.7s
 => => resolve docker.io/docker/dockerfile:1.5-labs@sha256:f2e91734a84c0922ff47aa4098ab775f1dfa932430d2888dd5cad5251fafdac4                                                                                                                     0.0s
 => => sha256:f2e91734a84c0922ff47aa4098ab775f1dfa932430d2888dd5cad5251fafdac4 8.40kB / 8.40kB                                                                                                                                                  0.0s
 => => sha256:2cee20f5babed260d08346a30bbbe7acc90b5e5a928278a25a5d4ba85706f08a 482B / 482B                                                                                                                                                      0.0s
 => => sha256:374405fa9fda27ab4568b5d9feaf72c2d81bd29b33a95199e8c1c2a5b1ac2e78 2.96kB / 2.96kB                                                                                                                                                  0.0s
 => => sha256:0bbf20a4d5d3f1f70cac395a6a777c95d971e8e1bc1084daf465ef47f5b71c1e 11.38MB / 11.38MB                                                                                                                                                0.5s
 => => extracting sha256:0bbf20a4d5d3f1f70cac395a6a777c95d971e8e1bc1084daf465ef47f5b71c1e                                                                                                                                                       0.1s
 => [internal] load metadata for registry.fedoraproject.org/fedora:38                                                                                                                                                                           0.6s
 => [ 1/11] FROM registry.fedoraproject.org/fedora:38@sha256:6349d2df6b4322c5690df1bb7743c45c356e20471dda69f27218cd9ba4a6c3c7                                                                                                                   4.7s
 => => resolve registry.fedoraproject.org/fedora:38@sha256:6349d2df6b4322c5690df1bb7743c45c356e20471dda69f27218cd9ba4a6c3c7                                                                                                                     0.0s
 => => sha256:6349d2df6b4322c5690df1bb7743c45c356e20471dda69f27218cd9ba4a6c3c7 955B / 955B                                                                                                                                                      0.0s
 => => sha256:4d6153e218ecb58fa61d991d8efaa41aa9c0577ee2c4f1cc31885802c43a0db8 429B / 429B                                                                                                                                                      0.0s
 => => sha256:ad336967d1c64583b2bc6f561e65533d09e03c05ea6f85982ce3b781ab735cc1 1.32kB / 1.32kB                                                                                                                                                  0.0s
 => => sha256:36e2634c79483a0414e4207eb995fca4283198b9be3259fbf2a9e4fc7d19dc1c 71.29MB / 71.29MB                                                                                                                                                2.2s
 => => extracting sha256:36e2634c79483a0414e4207eb995fca4283198b9be3259fbf2a9e4fc7d19dc1c                                                                                                                                                       2.3s
 => https://dl.google.com/go/go1.20.12.linux-amd64.tar.gz                                                                                                                                                                                       2.3s
 => https://github.com/mikefarah/yq/releases/download/v4.35.1/yq_linux_amd64                                                                                                                                                                    0.3s
 => https://sh.rustup.rs                                                                                                                                                                                                                        2.3s
 => https://github.com/protocolbuffers/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip                                                                                                                                        2.3s
 => [ 2/11] RUN dnf groupinstall -y 'Development Tools' &&     dnf install -y yum-utils gnupg git perl-core pkg-config libseccomp-devel gpgme-devel         device-mapper-devel unzip libassuan-devel         perl-FindBin openssl-devel tpm  107.7s
 => [ 3/11] ADD https://dl.google.com/go/go1.20.12.linux-amd64.tar.gz go1.20.12.linux-amd64.tar.gz                                                                                                                                              3.7s
 => [ 4/11] RUN rm -rf /usr/local/go && tar -C /usr/local -xzf go1.20.12.linux-amd64.tar.gz && rm -f go1.20.12.linux-amd64.tar.gz                                                                                                               7.3s
 => [ 5/11] ADD --checksum=sha256:bd695a6513f1196aeda17b174a15e9c351843fb1cef5f9be0af170f2dd744f08 https://github.com/mikefarah/yq/releases/download/v4.35.1/yq_linux_amd64 /usr/local/bin/yq                                                   0.4s
 => [ 6/11] RUN chmod a+x /usr/local/bin/yq                                                                                                                                                                                                     0.3s
 => [ 7/11] ADD https://sh.rustup.rs rustup                                                                                                                                                                                                     0.0s
 => [ 8/11] RUN chmod a+x rustup && ./rustup -y --default-toolchain 1.72.0                                                                                                                                                                     56.7s
 => [ 9/11] ADD https://github.com/protocolbuffers/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip protoc-3.11.4-linux-x86_64.zip                                                                                             0.0s
 => [10/11] RUN unzip protoc-3.11.4-linux-x86_64.zip -d /usr/local && rm -f protoc-3.11.4-linux-x86_64.zip                                                                                                                                      0.5s
 => [11/11] WORKDIR /src                                                                                                                                                                                                                        0.0s
 => exporting to image                                                                                                                                                                                                                         41.8s
 => => exporting layers                                                                                                                                                                                                                        41.8s
 => => writing image sha256:acd410112a898fa227acfc0be891e24f339279ad85292d96f731537ae351e572                                                                                                                                                    0.0s
 => => naming to docker.io/library/fedora-binaries-builder                                                                                                                                                                                      0.0s
root@liudali-s390x-libvirt16:~/cloud-api-adaptor/podvm-mkosi# docker run -it docker.io/library/fedora-binaries-builder bash
[root@7c047bba4e6a /]# uname -a
Linux 7c047bba4e6a 5.15.0-89-generic #99-Ubuntu SMP Mon Oct 30 20:42:29 UTC 2023 s390x GNU/Linux
[root@7c047bba4e6a /]# yq
bash: /usr/local/bin/yq: cannot execute binary file: Exec format error
[root@7c047bba4e6a /]# go
bash: /usr/local/go/bin/go: cannot execute binary file: Exec format error
[root@7c047bba4e6a /]# protoc
bash: /usr/local/bin/protoc: cannot execute binary file: Exec format error

lysliu

I think it's better to set values per arch,

cloud-api-adaptor/podvm/Dockerfile.podvm_builder.fedora

Lines 18 to 20 in cf5d30d

    
           # amd64: YQ_CHECKSUM="sha256:bd695a6513f1196aeda17b174a15e9c351843fb1cef5f9be0af170f2dd744f08" 
        
           # s390x: YQ_CHECKSUM="sha256:4e6324d08630e7df733894a11830412a43703682d65a76f1fc925aac08268a45" 
        
           ARG YQ_CHECKSUM="sha256:bd695a6513f1196aeda17b174a15e9c351843fb1cef5f9be0af170f2dd744f08"

and

cloud-api-adaptor/podvm/Dockerfile.podvm_builder.fedora

Lines 12 to 13 in cf5d30d

    
           # PROTOC_ARCH="x86_64" | "s390x_64" 
        
           ARG PROTOC_ARCH="x86_64"

otherwise, the file need updated before integrated into pipeline

lysliu · 2024-01-26T07:01:39Z

I tested the PR, set all the values according s390x arch. success to build out podvm binaries for s390x arch.

$ make binaries
docker buildx use default
Building binaries...
rm -rf ./resources/binaries-tree
docker buildx build \
	--build-arg BUILDER_IMG=fedora-binaries-builder \
	--build-arg AA_KBC=offline_fs_kbc \
	-o type=local,dest="./resources/binaries-tree" \
	-f ../podvm/Dockerfile.podvm_binaries.fedora ../.
[+] Building 1226.3s (10/10) FINISHED                                                                                                                                           docker:default
 => [internal] load .dockerignore                                                                                                                                                         0.0s
 => => transferring context: 254B                                                                                                                                                         0.0s
 => [internal] load build definition from Dockerfile.podvm_binaries.fedora                                                                                                                0.0s
 => => transferring dockerfile: 913B                                                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/fedora-binaries-builder:latest                                                                                                         0.0s
 => [internal] load build context                                                                                                                                                         0.0s
 => => transferring context: 38.92kB                                                                                                                                                      0.0s
 => CACHED [podvm_builder 1/4] FROM docker.io/library/fedora-binaries-builder                                                                                                             0.0s
 => [podvm_builder 2/4] COPY . /src/cloud-api-adaptor                                                                                                                                     0.1s
 => [podvm_builder 3/4] WORKDIR /src/cloud-api-adaptor/podvm                                                                                                                              0.0s
 => [podvm_builder 4/4] RUN LIBC=gnu make binaries                                                                                                                                     1220.3s
 => [stage-1 1/1] COPY --from=podvm_builder /src/cloud-api-adaptor/podvm/files /                                                                                                          4.8s
 => exporting to client directory                                                                                                                                                         0.5s
 => => copying files 239.91MB                                                                                                                                                             0.5s

huoqifeng · 2024-01-26T07:09:17Z

Thanks @huoqifeng I built out the docker.io/library/fedora-binaries-builder image on my s390x vsi, the image is built out but some require packages are BAD. Is this a support user case: build and run the fedora-builder container on s390x host directly?

root@liudali-s390x-libvirt16:~/cloud-api-adaptor/podvm-mkosi# make fedora-binaries-builder
Building fedora-binaries-builder image...
docker buildx build \
	-t fedora-binaries-builder \
	--load \
	-f ../podvm/Dockerfile.podvm_builder.fedora ../.
[+] Building 225.1s (21/21) FINISHED
 => [internal] load build definition from Dockerfile.podvm_builder.fedora                                                                                                                                                                       0.0s
 => => transferring dockerfile: 1.95kB                                                                                                                                                                                                          0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                               0.0s
 => => transferring context: 254B                                                                                                                                                                                                               0.0s
 => resolve image config for docker.io/docker/dockerfile:1.5-labs                                                                                                                                                                               0.6s
 => docker-image://docker.io/docker/dockerfile:1.5-labs@sha256:f2e91734a84c0922ff47aa4098ab775f1dfa932430d2888dd5cad5251fafdac4                                                                                                                 0.7s
 => => resolve docker.io/docker/dockerfile:1.5-labs@sha256:f2e91734a84c0922ff47aa4098ab775f1dfa932430d2888dd5cad5251fafdac4                                                                                                                     0.0s
 => => sha256:f2e91734a84c0922ff47aa4098ab775f1dfa932430d2888dd5cad5251fafdac4 8.40kB / 8.40kB                                                                                                                                                  0.0s
 => => sha256:2cee20f5babed260d08346a30bbbe7acc90b5e5a928278a25a5d4ba85706f08a 482B / 482B                                                                                                                                                      0.0s
 => => sha256:374405fa9fda27ab4568b5d9feaf72c2d81bd29b33a95199e8c1c2a5b1ac2e78 2.96kB / 2.96kB                                                                                                                                                  0.0s
 => => sha256:0bbf20a4d5d3f1f70cac395a6a777c95d971e8e1bc1084daf465ef47f5b71c1e 11.38MB / 11.38MB                                                                                                                                                0.5s
 => => extracting sha256:0bbf20a4d5d3f1f70cac395a6a777c95d971e8e1bc1084daf465ef47f5b71c1e                                                                                                                                                       0.1s
 => [internal] load metadata for registry.fedoraproject.org/fedora:38                                                                                                                                                                           0.6s
 => [ 1/11] FROM registry.fedoraproject.org/fedora:38@sha256:6349d2df6b4322c5690df1bb7743c45c356e20471dda69f27218cd9ba4a6c3c7                                                                                                                   4.7s
 => => resolve registry.fedoraproject.org/fedora:38@sha256:6349d2df6b4322c5690df1bb7743c45c356e20471dda69f27218cd9ba4a6c3c7                                                                                                                     0.0s
 => => sha256:6349d2df6b4322c5690df1bb7743c45c356e20471dda69f27218cd9ba4a6c3c7 955B / 955B                                                                                                                                                      0.0s
 => => sha256:4d6153e218ecb58fa61d991d8efaa41aa9c0577ee2c4f1cc31885802c43a0db8 429B / 429B                                                                                                                                                      0.0s
 => => sha256:ad336967d1c64583b2bc6f561e65533d09e03c05ea6f85982ce3b781ab735cc1 1.32kB / 1.32kB                                                                                                                                                  0.0s
 => => sha256:36e2634c79483a0414e4207eb995fca4283198b9be3259fbf2a9e4fc7d19dc1c 71.29MB / 71.29MB                                                                                                                                                2.2s
 => => extracting sha256:36e2634c79483a0414e4207eb995fca4283198b9be3259fbf2a9e4fc7d19dc1c                                                                                                                                                       2.3s
 => https://dl.google.com/go/go1.20.12.linux-amd64.tar.gz                                                                                                                                                                                       2.3s
 => https://github.com/mikefarah/yq/releases/download/v4.35.1/yq_linux_amd64                                                                                                                                                                    0.3s
 => https://sh.rustup.rs                                                                                                                                                                                                                        2.3s
 => https://github.com/protocolbuffers/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip                                                                                                                                        2.3s
 => [ 2/11] RUN dnf groupinstall -y 'Development Tools' &&     dnf install -y yum-utils gnupg git perl-core pkg-config libseccomp-devel gpgme-devel         device-mapper-devel unzip libassuan-devel         perl-FindBin openssl-devel tpm  107.7s
 => [ 3/11] ADD https://dl.google.com/go/go1.20.12.linux-amd64.tar.gz go1.20.12.linux-amd64.tar.gz                                                                                                                                              3.7s
 => [ 4/11] RUN rm -rf /usr/local/go && tar -C /usr/local -xzf go1.20.12.linux-amd64.tar.gz && rm -f go1.20.12.linux-amd64.tar.gz                                                                                                               7.3s
 => [ 5/11] ADD --checksum=sha256:bd695a6513f1196aeda17b174a15e9c351843fb1cef5f9be0af170f2dd744f08 https://github.com/mikefarah/yq/releases/download/v4.35.1/yq_linux_amd64 /usr/local/bin/yq                                                   0.4s
 => [ 6/11] RUN chmod a+x /usr/local/bin/yq                                                                                                                                                                                                     0.3s
 => [ 7/11] ADD https://sh.rustup.rs rustup                                                                                                                                                                                                     0.0s
 => [ 8/11] RUN chmod a+x rustup && ./rustup -y --default-toolchain 1.72.0                                                                                                                                                                     56.7s
 => [ 9/11] ADD https://github.com/protocolbuffers/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip protoc-3.11.4-linux-x86_64.zip                                                                                             0.0s
 => [10/11] RUN unzip protoc-3.11.4-linux-x86_64.zip -d /usr/local && rm -f protoc-3.11.4-linux-x86_64.zip                                                                                                                                      0.5s
 => [11/11] WORKDIR /src                                                                                                                                                                                                                        0.0s
 => exporting to image                                                                                                                                                                                                                         41.8s
 => => exporting layers                                                                                                                                                                                                                        41.8s
 => => writing image sha256:acd410112a898fa227acfc0be891e24f339279ad85292d96f731537ae351e572                                                                                                                                                    0.0s
 => => naming to docker.io/library/fedora-binaries-builder                                                                                                                                                                                      0.0s
root@liudali-s390x-libvirt16:~/cloud-api-adaptor/podvm-mkosi# docker run -it docker.io/library/fedora-binaries-builder bash
[root@7c047bba4e6a /]# uname -a
Linux 7c047bba4e6a 5.15.0-89-generic #99-Ubuntu SMP Mon Oct 30 20:42:29 UTC 2023 s390x GNU/Linux
[root@7c047bba4e6a /]# yq
bash: /usr/local/bin/yq: cannot execute binary file: Exec format error
[root@7c047bba4e6a /]# go
bash: /usr/local/go/bin/go: cannot execute binary file: Exec format error
[root@7c047bba4e6a /]# protoc
bash: /usr/local/bin/protoc: cannot execute binary file: Exec format error

To run the builder build:

docker buildx build \
	-t fedora-binaries-builder-s390x \
	--build-arg ARCH=s390x \
	--build-arg PROTOC_ARCH=s390x_64 \
	--build-arg YQ_ARCH=s390x \
	--build-arg YQ_CHECKSUM=sha256:4e6324d08630e7df733894a11830412a43703682d65a76f1fc925aac08268a45 \
	--load \
	-f podvm/Dockerfile.podvm_builder.fedora .

To run the binaries build against the builder image

docker buildx build \
	--build-arg BUILDER_IMG=fedora-binaries-builder-s390x:latest \
	--build-arg AA_KBC=offline_fs_kbc \
	-o type=local,dest="./resources/binaries-tree" \
	-f podvm/Dockerfile.podvm_binaries.fedora .

@liudalibj s390x is not enabled yet in CI because we don't have s390x runner yet, this PR just make changes for Dockerfile/Makefile.

liudalibj · 2024-01-28T03:52:04Z

docker buildx build
--build-arg BUILDER_IMG=fedora-binaries-builder-s390x:latest
--build-arg AA_KBC=offline_fs_kbc
-o type=local,dest="./resources/binaries-tree"
-f podvm/Dockerfile.podvm_binaries.fedora .

With the provider args, I can built out images on s390x host with expected result:

root@liudali-s390x-libvirt:~/cloud-api-adaptor# docker buildx build \
        --build-arg BUILDER_IMG=liudali/fedora-binaries-builder-s390x:latest \
        --build-arg AA_KBC=offline_fs_kbc \
        -o type=local,dest="./resources/binaries-tree" \
        -f podvm/Dockerfile.podvm_binaries.fedora .

[+] Building 1519.6s (11/11) FINISHED                                                                                                                                                                                 docker-container:multi-builder
 => [internal] load build definition from Dockerfile.podvm_binaries.fedora                                                                                                                                                                      0.0s
 => => transferring dockerfile: 913B                                                                                                                                                                                                            0.0s
 => [internal] load metadata for docker.io/liudali/fedora-binaries-builder-s390x:latest                                                                                                                                                         1.4s
 => [auth] liudali/fedora-binaries-builder-s390x:pull token for registry-1.docker.io                                                                                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                               0.0s
 => => transferring context: 254B                                                                                                                                                                                                               0.0s
 => [internal] load build context                                                                                                                                                                                                               1.5s
 => => transferring context: 195.70MB                                                                                                                                                                                                           1.5s
 => CACHED [podvm_builder 1/4] FROM docker.io/liudali/fedora-binaries-builder-s390x:latest@sha256:b570ed69a8e9f4a43659fac36913e550d9379cf3b28091fb80a3af8a5a08e378                                                                              0.0s
 => => resolve docker.io/liudali/fedora-binaries-builder-s390x:latest@sha256:b570ed69a8e9f4a43659fac36913e550d9379cf3b28091fb80a3af8a5a08e378                                                                                                   0.0s
 => [podvm_builder 2/4] COPY . /src/cloud-api-adaptor                                                                                                                                                                                           7.8s
 => [podvm_builder 3/4] WORKDIR /src/cloud-api-adaptor/podvm                                                                                                                                                                                    0.1s
 => [podvm_builder 4/4] RUN LIBC=gnu make binaries                                                                                                                                                                                           1507.1s
 => [stage-1 1/1] COPY --from=podvm_builder /src/cloud-api-adaptor/podvm/files /                                                                                                                                                                0.2s
 => exporting to client directory                                                                                                                                                                                                               0.9s
 => => copying files 195.67MB

root@liudali-s390x-libvirt:~/cloud-api-adaptor/resources/binaries-tree# tree
.
├── etc
│   ├── aa-offline_fs_kbc-keys.json
│   ├── aa-offline_fs_kbc-resources.json
│   ├── agent-config.toml
│   ├── certificates
│   │   └── _add_certs_
│   ├── containers
│   │   └── policy.json
│   └── systemd
│       └── system
│           ├── agent-protocol-forwarder.service
│           ├── api-server-rest.service
│           ├── kata-agent.service
│           ├── multi-user.target.wants
│           │   ├── agent-protocol-forwarder.service -> ../agent-protocol-forwarder.service
│           │   ├── api-server-rest.service -> ../api-server-rest.service
│           │   ├── kata-agent.service -> ../kata-agent.service
│           │   ├── process-user-data.service -> ../process-user-data.service
│           │   ├── run-image.mount -> ../run-image.mount
│           │   └── run-kata\x2dcontainers.mount -> ../run-kata\x2dcontainers.mount
│           ├── netns@.service
│           ├── process-user-data.service
│           ├── run-image.mount
│           ├── run-kata\x2dcontainers.mount
│           └── setup-nat-for-imds.service
├── pause_bundle
│   ├── config.json
│   ├── rootfs
│   │   └── pause
│   ├── sha256_575902b3a43344546014db9a4f05fcc946afc8ce03009a6d0afb06df703dadc3.mtree
│   └── umoci.json
└── usr
    └── local
        └── bin
            ├── agent-protocol-forwarder
            ├── api-server-rest
            ├── attestation-agent
            ├── confidential-data-hub
            ├── kata-agent
            ├── kata-agent-clean
            ├── process-user-data
            └── setup-nat-for-imds.sh

11 directories, 31 files

file usr/local/bin/kata-agent
usr/local/bin/kata-agent: ELF 64-bit MSB pie executable, IBM S/390, version 1 (SYSV), dynamically linked, interpreter /lib/ld64.so.1, BuildID[sha1]=5898f458479137333e2a8a2613f16c9dbe57584e, for GNU/Linux 3.2.0, with debug_info, not stripped
file usr/local/bin/agent-protocol-forwarder
usr/local/bin/agent-protocol-forwarder: ELF 64-bit MSB executable, IBM S/390, version 1 (SYSV), statically linked, Go BuildID=ALUX8lVxoTnbRrVcuB9d/F2daRN79GlyMaI1gisOg/ySn9tcmaIaVsuPeewZnt/1po_yM7xGVcmu91E5C2Y, with debug_info, not stripped
file usr/local/bin/process-user-data
usr/local/bin/process-user-data: ELF 64-bit MSB executable, IBM S/390, version 1 (SYSV), statically linked, Go BuildID=MjnAikvU7NhkFUD8TkJj/t3yZZ7eFKkq5NFWFGfUz/GZriPKwzz5X4pWgX4tCX/8qw0gKPFmDUoi3b1JlV8, with debug_info, not stripped
...

liudalibj

LGTM, thanks @huoqifeng

Fixes: confidential-containers#1640 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

huoqifeng · 2024-01-29T00:44:01Z

Makefile in podvm/mkosi is revised, so that the command can be run successfully both on x86 and s390x

cd podvm-mkosi
make fedora-binaries-builder
make binaries

s390x: build fedora binaries for peerpod

cf5d30d

Fixes: confidential-containers#1640 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

huoqifeng marked this pull request as ready for review January 26, 2024 04:54

huoqifeng requested review from katexochen, wainersm, liudalibj, genjuro214 and stevenhorsman January 26, 2024 05:00

This comment was marked as duplicate.

Sign in to view

lysliu suggested changes Jan 26, 2024

View reviewed changes

liudalibj approved these changes Jan 28, 2024

View reviewed changes

s390x: revise Makefile for building fedora binaries

f59c8bf

Fixes: confidential-containers#1640 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

genjuro214 approved these changes Jan 29, 2024

View reviewed changes

huoqifeng merged commit 59e5bc6 into confidential-containers:main Jan 29, 2024
18 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

s390x: build fedora binaries for peerpod #1678

s390x: build fedora binaries for peerpod #1678

huoqifeng commented Jan 26, 2024 •

edited

huoqifeng commented Jan 26, 2024 •

edited

liudalibj commented Jan 26, 2024 •

edited

This comment was marked as duplicate.

lysliu left a comment

lysliu commented Jan 26, 2024

huoqifeng commented Jan 26, 2024 •

edited

liudalibj commented Jan 28, 2024

liudalibj left a comment

huoqifeng commented Jan 29, 2024 •

edited

	# amd64: YQ_CHECKSUM="sha256:bd695a6513f1196aeda17b174a15e9c351843fb1cef5f9be0af170f2dd744f08"
	# s390x: YQ_CHECKSUM="sha256:4e6324d08630e7df733894a11830412a43703682d65a76f1fc925aac08268a45"
	ARG YQ_CHECKSUM="sha256:bd695a6513f1196aeda17b174a15e9c351843fb1cef5f9be0af170f2dd744f08"

s390x: build fedora binaries for peerpod #1678

s390x: build fedora binaries for peerpod #1678

Conversation

huoqifeng commented Jan 26, 2024 • edited

huoqifeng commented Jan 26, 2024 • edited

liudalibj commented Jan 26, 2024 • edited

This comment was marked as duplicate.

lysliu left a comment

Choose a reason for hiding this comment

lysliu commented Jan 26, 2024

huoqifeng commented Jan 26, 2024 • edited

liudalibj commented Jan 28, 2024

liudalibj left a comment

Choose a reason for hiding this comment

huoqifeng commented Jan 29, 2024 • edited

huoqifeng commented Jan 26, 2024 •

edited

huoqifeng commented Jan 26, 2024 •

edited

liudalibj commented Jan 26, 2024 •

edited

huoqifeng commented Jan 26, 2024 •

edited

huoqifeng commented Jan 29, 2024 •

edited