New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide ARMv6l binaries #5525
Provide ARMv6l binaries #5525
Conversation
ARM is a quite popular for small computing clusters, and an interesting pet project to try Nomad on. Unfortunately, the current ARM builds don't run on Raspberry Pi's 1, Zero or Zero W as it is compiled with some operations not available there. When executing the current binary, you receive this message: ``` Illegal instruction (core dumped) ``` The difference is caused by the version of the ARM instructions are available on each version of the Pi. | Version | Compiler arch | |-------------+----------------------------| | Pi v1, Zero | armel, floating point (v6) | | Pi v2, v3 | armhf, floating point (v7) | Fortunately, cross-compiling using Go is not so troublesome. It is possible to produce an executable using the non-hf version of `gcc`. This commit introduces a new `make` target for armel binaries. It renames the previous one so it is clear the difference between artifacts versions. A couple of extra compilation dependencies are needed on the build machine. Tested `armel` o a Pi Zero W 1.3 and Pi 3, which executes, but there are not yet any jobs running. The `armel` binary should work on all Raspberry Pi models, while the `armhf` only works on Pi 2/3 and it is optimized. Fixes hashicorp#2517
I was able to test this on a local cluster, using a mix of Pi Zero, Pi3 and amd64 machines and got it running the Redis deployment example. If it was not for the bugs [1][2] on Docker image selection logic, it would have worked on the first allocation try. Nomad already provides the processor architecture, but it does not include the variant. I've then added two meta tags on each consul client, to help selecting the image to run on each node based on the variant: client {
enabled = true
node_class = "armv6l" # value from $(uname -m)
meta {
"arch" = "armv6l" # value from $(uname -m)
"arch_base" = "arm32v6" # mapping logic to Docker multiarch repos prefix (eg: arm32v6, amd64, arm64v8...)
}
} With that, I was able to run the redis example: job "redis" {
datacenters = ["dc1"]
constraint {
attribute = "${attr.kernel.name}"
value = "linux"
}
update {
stagger = "10s"
max_parallel = 1
}
group "cache" {
count = 3
restart {
attempts = 2
interval = "1m"
delay = "10s"
mode = "fail"
}
task "redis" {
driver = "docker"
config {
image = "${meta.arch_base}/redis:alpine" # arm32v6/redis does not have latest tag
port_map {
db = 6379
}
}
service {
name = "${TASKGROUP}-redis"
tags = ["global", "cache"]
port = "db"
check {
name = "alive"
type = "tcp"
interval = "10s"
timeout = "2s"
}
}
resources {
cpu = 500 # 500 MHz
memory = 256 # 256MB
network {
mbits = 10
port "db" {
}
}
}
}
}
} Precompiled binaries here: https://github.com/bltavares/nomad/releases/tag/v0.9.0-armel |
Hi @bltavares , thank you for submitting this. However, we are going to leave the older ARM architectures up to community support. A preferred avenue for supporting these (and other niche) platforms will be external repos like @nicholasjackson https://github.com/nicholasjackson/nomad-arm6l. |
@cgbaker Thanks for letting us know. Its unfortunate that The suggested project, unfortunately, produces a Nomad binary which does not work when scheduling tasks. It fails due to to some cross-compilation issues I haven't been able to resolve yet (it's related to how The binary compile with these changes do work tho. I could try to configure a new Docker builder with the same setup as the Vagrant machine as an alternative project, but I'll most likely just keep my fork updated then. |
I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions. |
ARM is a quite popular for small computing clusters, and an interesting
pet project to try Nomad on.
Unfortunately, the current ARM builds don't run on Raspberry Pi's 1,
Zero or Zero W as it is compiled with some operations not available
there.
When executing the current binary, you receive this message:
The difference is caused by the version of the ARM instructions are
available on each version of the Pi.
| Version | Compiler arch |
|-------------+----------------------------|
| Pi v1, Zero | armel, floating point (v6) |
| Pi v2, v3 | armhf, floating point (v7) |
Fortunately, cross-compiling using Go is not so troublesome. It is possible to
produce an executable using the non-hf version of
gcc
.This commit introduces a new
make
target for armel binaries. It renames theprevious one so it is clear the difference between artifacts versions.
A couple of extra compilation dependencies are needed on the build machine.
Tested
armel
o a Pi Zero W 1.3 and Pi 3, which executes, but there are not yetany jobs running. The
armel
binary should work on all Raspberry Pi models,while the
armhf
only works on Pi 2/3 and it is optimized.Fixes #2517