-
Notifications
You must be signed in to change notification settings - Fork 6.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Track the gaps when porting to ARM (arm7l) #4294
Comments
Also default pause container is invalid (( plus has '\n' in it for arm7l and it breaks systemd unit )) |
works for me |
what are the production usecase to run workload on arm7 ? |
IoT on BBB devices in my usecase for example |
(( Beagle bone black )) |
what can of load are you running on those machines? why kubernetes? |
A quite simple load, reading from serial UARTs and sending to the message queue data.
Because I'm familiar with it and it serves me health checks, deployments, upgrade process nicely. It nicely manages secrets and monitoring. With other solutions such as consul + ansible + docker I'd had to have some verification deployment successfully completed + gradual rollout. Maybe I could also use a spinnaker or something like that, though I'm not familiar with it, and I run k8s for the rest of the infrastructure. K8s downside on edge devices in CPU usage...it's around 15% CPU time just on kubelet, even after tuning various housekeeping/node status freq parameters. Most of the time is spent during syscalls, runtime, and some JSON decoding. EDIT: This is AM335x 1GHz ARM® Cortex-A8, and I see 30% branch misprediction rate system-wide ( also similar amount for kubelet )...so yeah, not the best processor in the world nor the most powerful. |
According to coreOS docs
Etcd-io doesn't provide any 32bits ARM binaries because of a Go language issue. Otherwise we could have downloaded tarballs from cores/etcd-io (https://github.com/coreos/etcd/releases/download/) and checksums . |
/kind feature |
@Miouge1: Please ensure the request meets the requirements listed here. If this request no longer meets these requirements, the label can be removed In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
I ran into this issue today. I have a esxi home lab with limited amount of RAM so I thought it would be nice to run masters / etcd on ARM and minions on vm's. etcd seems to be the only blocker. |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
/remove-lifecycle stale |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
Stale issues rot after 30d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
@rich-nahra well etcd works on arm64, but not on arm32. I've seen lots of people running k3s on arm32 (raspberry pi) which uses sqlite instead of etcd, so I guess that's a possible work around for test labs. @lwolf and @nmiculinic I would consider "etcd for arm32" out of scope of Kubespray, is there anything else we can do to make life easier for Kubespray on ARM 32bits? |
@Miouge1 I agree that etcd is probably out of scope. I recently migrated my arm32 cluster to k3s. I need to check if it still relevant, but last time I check this one was still an issue - 2.Add NodeSelector to some manifests |
There's no calico release for arm (32bit) too, is there a work around for that too ? (I have a seperate 64bit etcd node to get around the lack of a 32bit etcd) |
The only CNI that works on arm32 at the moment is flannel. |
Well, arm is supported, there are just no builds in the official repo and its flagged as unstable: https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/supported-platform.md For example Ubuntu has it in the official repo: https://packages.ubuntu.com/search?keywords=etcd But it seems to require an experimental flag set - its unstable due to a bug in Go, that's existing 9 years now. Funnily speaking just recently it picked up tracktion and there was a prototype implemented to fix this bug for 32 bit systems: That said we're speaking about armhf only here so armv7+ with hardware floating point unit - no other architectures likely possible (???) That includes the Raspberry Pi 3 Pre-B or the Odroid XU4-Platform (e.g. Odroid HC1) Anyway it runs on Ubuntu Bionic Beaver 18.04 on the Odroid HC1 for me with the flag mentioned set in the systemd.service:
|
Rotten issues close after 30d of inactivity. Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
@fejta-bot: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/reopen |
@hadrien-toma: You can't reopen an issue/PR unless you authored it or you are a collaborator. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/reopen |
@lwolf: Reopened this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Rotten issues close after 30d of inactivity. Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
@fejta-bot: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Is this a BUG REPORT or FEATURE REQUEST? (choose one):
FEATURE REQUEST
Environment:
Cloud provider or hardware configuration:
Hardware
OS (
printf "$(uname -srm)\n$(cat /etc/os-release)\n"
):ansible --version
):ansible 2.7.2
python version = 2.7.10 (default, Oct 6 2017, 22:29:07) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
Kubespray version (commit) (
git rev-parse --short HEAD
): Latest master branchAnything else do we need to know:
At the moment it is impossible to install kubespray on pure arm7l hardware.
Most of the containers does not provide binaries/containers for non-amd64 arch.
The aim of this ticket is to make it possible use ARM hardware as pool of worker nodes alongside the amd64 master/nodes.
I've found a several gaps when trying to install kubespray on arm7l devices. So
related issues: #4261 #4065
The text was updated successfully, but these errors were encountered: