Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support running on aarch64 arch #26

Closed
5 of 6 tasks
innobead opened this issue Sep 20, 2020 · 9 comments
Closed
5 of 6 tasks

Support running on aarch64 arch #26

innobead opened this issue Sep 20, 2020 · 9 comments
Labels
enhancement New feature or request

Comments

@innobead
Copy link
Owner

innobead commented Sep 20, 2020

Is your feature request related to a problem? Please describe.
Support using kubefire on aarch64 arch.

Describe the solution you'd like

  • have rootfs images for aarch64
  • have kernel images for aarch64
  • install dependent components for aarch64 (ex: containerd, runc, cni plugins, firecracker prepared by ignite)
    • install-prerequisites.sh
    • uninstall-prerequisites.sh
  • only allow to use k3s bootstrapper for aarch64

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

@innobead innobead added the enhancement New feature or request label Sep 20, 2020
@innobead
Copy link
Owner Author

innobead commented Sep 22, 2020

https://docs.docker.com/docker-for-mac/multi-arch/
weaveworks/ignite#574
https://github.com/weaveworks/ignite/blob/master/docs/arm.md

Ignite depends on Firecracker's ARM 64-bit support. Ignite and Firecracker are currently built for arm64, also known as aarch64.

@innobead
Copy link
Owner Author

Example

10082  docker buildx use mybuilder
10083  docker buildx inspect --bootstrap
10084  docker buildx ls
10085  cd github/innobead/kubefire
10086  docker buildx ls
10087  docker buildx build --platform linux/arm64,linux/amd64 --build-arg="RELEASE=20.04" -t ubuntu-local:20.04 ./build/images/ubuntu/
10088  docker buildx build --platform linux/arm64,linux/amd64 --build-arg="RELEASE=20.04" -t ubuntu-local:20.04 -f /home/davidko/github/innobead/kubefire/build/images/ubuntu/Dockerfile .
10089  docker buildx
10090  docker images
10091  docker images | grep ubuntu
10092  docker buildx imagetools
10093  docker buildx imagetools inpsect ubuntu-local:20.04
10094  docker buildx build --platform linux/arm64,linux/amd64 --build-arg="RELEASE=20.04" -t ubuntu-local:20.04 -f /home/davidko/github/innobead/kubefire/build/images/ubuntu/Dockerfile .
10095  docker buildx build --platform linux/arm64,linux/amd64 --build-arg="RELEASE=20.04" -t ubuntu-local:20.04 --load -f /home/davidko/github/innobead/kubefire/build/images/ubuntu/Dockerfile .
10096  docker buildx build --platform linux/arm64,linux/amd64 --build-arg="RELEASE=20.04" -t innobead/kubefire-ubuntu:20.04 --push -f /home/davidko/github/innobead/kubefire/build/images/ubuntu/Dockerfile .
10097  docker buildx imagetools inspect  docker.io/innobead/kubefire-ubuntu:20.04
10098  docker run docker.io/innobead/kubefire-ubuntu:20.04@sha256:935a0c3e0b9f1ca3b10acb9b6c1a1369ef8b65bed6585ecbb4545cafb03e317f
10099  docker run docker.io/innobead/kubefire-ubuntu:20.04@sha256:935a0c3e0b9f1ca3b10acb9b6c1a1369ef8b65bed6585ecbb4545cafb03e317f -- uname -a
10100  docker run docker.io/innobead/kubefire-ubuntu:20.04@sha256:935a0c3e0b9f1ca3b10acb9b6c1a1369ef8b65bed6585ecbb4545cafb03e317f uname -a
10101  docker run docker.io/innobead/kubefire-ubuntu:20.04@sha256:993137f5b10535b700e236e035e344859d1eaa237972b2ff88cd0f7e2a9f6ae5 uname -a

@innobead
Copy link
Owner Author

~ docker run --privileged --rm tonistiigi/binfmt --install all
Unable to find image 'tonistiigi/binfmt:latest' locally
latest: Pulling from tonistiigi/binfmt
9cc57093ca8c: Pull complete 
b78aa434a304: Pull complete 
Digest: sha256:df15403e06a03c2f461c1f7938b171fda34a5849eb63a70e2a2109ed5a778bde
Status: Downloaded newer image for tonistiigi/binfmt:latest
2020/09/22 16:03:53 installing: riscv64 OK
{
  "supported": [
    "linux/amd64",
    "linux/arm64",
    "linux/riscv64",
    "linux/ppc64le",
    "linux/s390x",
    "linux/386",
    "linux/arm/v7",
    "linux/arm/v6"
  ],
  "emulators": [
    "qemu-aarch64",
    "qemu-arm",
    "qemu-ppc64le",
    "qemu-riscv64",
    "qemu-s390x"
  ]
}

@innobead
Copy link
Owner Author

  ~ docker buildx ls                              
NAME/NODE  DRIVER/ENDPOINT       STATUS PLATFORMS
mybuild    docker-container             
  mybuild0 tcp://node-amd64:2375 error during connect: Get http://node-amd64:2375/v1.24/containers/buildx_buildkit_mybuild0/json: dial tcp: lookup node-amd64 on 192.168.1.1:53: no such host
default *  docker                        
  default  default               running linux/amd64, linux/386

@innobead
Copy link
Owner Author

innobead commented Sep 22, 2020

kubefire (master) ✗ docker run docker.io/innobead/kubefire-ubuntu:20.04@sha256:935a0c3e0b9f1ca3b10acb9b6c1a1369ef8b65bed6585ecbb4545cafb03e317f uname -a
Linux 37c4b2e9fde9 5.3.18-lp152.36-default #1 SMP Tue Aug 18 17:09:44 UTC 2020 (885251f) x86_64 x86_64 x86_64 GNU/Linuxkubefire (master) ✗ docker run docker.io/innobead/kubefire-ubuntu:20.04@sha256:993137f5b10535b700e236e035e344859d1eaa237972b2ff88cd0f7e2a9f6ae5 uname -a
Linux 4c9997a6b2e6 5.3.18-lp152.36-default #1 SMP Tue Aug 18 17:09:44 UTC 2020 (885251f) aarch64 aarch64 aarch64 GNU/Linux

@innobead
Copy link
Owner Author

innobead commented Sep 22, 2020

Right now, ignite only supports building images based on the native arch instead of buildx.

https://github.com/weaveworks/ignite/blob/83ae8f5e62d603187c05617c363a7bfa32bbcaa0/images/kernel/Makefile#L57-L63

build-%:
	$(DOCKER) build -t $(IMAGE_NAME):$*-${GOARCH} \
		--build-arg KERNEL_VERSION=$* \
		--build-arg ARCH=${KERNEL_ARCH} \
		--build-arg GOARCH=${GOARCH} \
		--build-arg ARCH_MAKE_PARAMS=${ARCH_MAKE_PARAMS} \
		--build-arg VMLINUX_PATH=${VMLINUX_PATH} .

innobead added a commit that referenced this issue Sep 23, 2020
innobead added a commit that referenced this issue Sep 23, 2020
innobead added a commit that referenced this issue Sep 23, 2020
innobead added a commit that referenced this issue Sep 23, 2020
innobead added a commit that referenced this issue Sep 23, 2020
@innobead
Copy link
Owner Author

innobead commented Sep 23, 2020

containerd and runc do not provide arm prebuilt artifacts due to cgo dependencies util now.

But arch does provide, need to think if they are able to reuse as prebuilt artifacts.
ref:

Updated:
archlinux arm only has the latest version.

Solution:
Leave user to install via system package manager.

innobead added a commit that referenced this issue Sep 23, 2020
innobead added a commit that referenced this issue Sep 23, 2020
@innobead innobead changed the title Support running on ARM arch Support running on ARM64/aarch64 arch Sep 23, 2020
@innobead innobead reopened this Sep 23, 2020
innobead added a commit that referenced this issue Sep 23, 2020
innobead added a commit that referenced this issue Sep 23, 2020
@innobead
Copy link
Owner Author

Also, right now, only k3s boostrapper is able to support aarch64, because kubeadm needs containerd and runc, the same problem about required aarch64 prebuilt packages.

@innobead innobead changed the title Support running on ARM64/aarch64 arch Support running on aarch64 arch Sep 23, 2020
innobead added a commit that referenced this issue Sep 24, 2020
innobead added a commit that referenced this issue Sep 24, 2020
innobead added a commit that referenced this issue Sep 24, 2020
innobead added a commit that referenced this issue Sep 24, 2020
innobead added a commit that referenced this issue Sep 24, 2020
innobead added a commit that referenced this issue Sep 24, 2020
@innobead
Copy link
Owner Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant