Official ARM builds

[markwalkom]

Describe the enhancement: This has been asked on https://discuss.elastic.co a few times, I wanted to raise this here as well. Official ARM builds would be super useful for many different use cases.

Describe a specific use case for the enhancement or feature: I have a bunch of Ubiquiti network gear that all runs ARM, it'd be great to be able to deploy beats on them to easily get stats.
There are also vendors, like Scaleways or even now AWS, that offer ARM based servers so being able to deploy there makes sense.

[legrego]

What steps would be required to provide official ARM builds? I'd be happy to support this effort if I can

I need this to monitor all my Raspberry Pi's...

Either that or the correct instructions to cross-compile it myself :)

TIA!

[njbmartin]

Looking through the PRs and snapshots, it looks like the beats are already being built for ARM. However, they are not being published. It's that step which we require in order to use it.

Following these instructions on a Raspberry Pi:
https://www.elastic.co/guide/en/beats/metricbeat/current/setup-repositories.html

We get the following message:

$ sudo apt-get update
...
Reading package lists... Done
N: Skipping acquire of configured file 'main/binary-armhf/Packages' as repository 'https://artifacts.elastic.co/packages/7.x/apt stable InRelease' doesn't support architecture 'armhf'

[markwalkom]

Don't spose this is likely to happen anytime soon? It'd be super handy and awesome :)

[josh-thurston]

I have been reading through all of the various 'How To' guides scattered on the internet. Most are old and have errors when running with newer OS & software versions. I estimate that I have spent 20+ hours trying to get this working for two of my Raspberry Pi's with zero success so far. A lot of folks have spent time figuring this out, posting 'How To' guides, but those guides have short lifespans.

With the popularity of ARM these days, I don't understand why it is even a question for elastic to build, publish, and support.

[mikemadden42]

@josephgaldino Thanks for reaching out. I am also running into issues building beats on ARM. What specific issues are you running into?

I’ve been doing this on my Raspberry Pi 3 Model B Plus. Unfortunately it only has 1 GB of RAM and runs into out of memory issues during the build.

[2413110.529707] Out of memory: Kill process 9553 (link) score 341 or sacrifice child
[2413110.537889] Killed process 9553 (link) total-vm:380184kB, anon-rss:320568kB, file-rss:0kB, shmem-rss:0kB
[2413110.638338] oom_reaper: reaped process 9553 (link), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

I think it may be more fruitful to cross compile for ARM instead of building on resource contrained devices. Our official builds for beats rely on the following project.

https://github.com/elastic/golang-crossbuild

We only publish artifacts for i386 and x86_64, but I will check if snapshot builds include other architectures like ARM.

Supporting ARM is definitely on our radar. We need to invest some time setting up and testing build and CI services for ARM.

[maxviper]

I managed to create ARM build for filebeat, metricbeat, heartbeat and auditbeat, based on https://github.com/caglar10ur/elk-usg/#build-beats-for-mips64-and-put-them-under-elk-usg
however, I'm unable to do this for packetbeat. I already tried the https://github.com/elastic/golang-crossbuild but without any success. I'm not a specialist in this matter and would appreciate being able to download the different ARM and MIPS builds, rather than trying to build them myself. So please public ARM and MIPS build for beats

[mikemadden42]

I agree that it would be ideal to have beats artifacts available for all our supported platforms. I'll continue this discussion internally to assess its feasibility.

In the meantime, I've had decent success running the following command to build packages for other platforms.

PLATFORMS='+linux/armv7 +linux/arm64 +linux/mips +linux/mips64' make release-manager-release

This will produce packages built for the normal architectures like i386 and x86_64, and it will also build packages for armv7, arm64, mips, and mips64.

All the supported platforms can be found here.

I've verified it produces packages for the following beats. I'm running into build issues for packetbeat, and I have not tried winlogbeat yet.

• auditbeat
• filebeat
• heartbeat
• journalbeat
• metricbeat
• packetbeat
• winlogbeat

NOTE: This make target requires a development environment already setup for beats. This includes tools like make, go 1.12.x, mage, python 2.7.x, virtualenv, docker, and docker-compose.

More information about setting up a development environment for beats can be found at:
https://www.elastic.co/guide/en/beats/devguide/current/beats-contributing.html#setting-up-dev-environment

[josh-thurston]

@maxviper
I just finished up a script that will run on Raspberry Pi and create filebeat.

https://github.com/josh-thurston/pibeat

Hope it works for you. In the next day or so, i will update to create the rest of the beats packages.

[jbagel2]

With AWS having launched ARM instance types. I think there will be more and more interest in this. Hoping there may be a status update on this one? We currently just have a jenkins pipeline configured to rebuild each time a new version drops. but us doing that, is no better than you guys doing that for us and just giving it a alpha/beta designation. We are using it anyway...

[mvaldes14]

Need beats for my IOT devices 👍

[devinbfergy]

This would be great!

[josh-thurston]

Updated my script. Creating more than just filebeat for arm architecture now.

https://github.com/josh-thurston/easyBEATS

[FreerideTheWorld]

+1 on this topic
I'm currently trying to replicate a portable Elasticsearch "work" environment that people could learn on with 5 Raspberry Pi 4b's and one LattePanda with Windows installed. The entire setup can be housed in a C4 labs case and its portability would allow me to teach people with a more hands on and visual experience.

I appreciate those of you who have put the time into creating elasticsearch/beats that run on ARM as much of this is way past my knowledge base. Its taken me quite some time to figure out how to get ES running on Pi and that was only due to the fact that someone had posted a very useful guide in the discussion forums. Even then I had to put a few different guides together, quit trying to install 7.5 and just install 7.2. Kibana I didn't even attempt on Pi due to its potential difficulty and I just installed it on the LattePanda without any problems.