ARM builds

[TheAssassin]

Experimental ARM builds (armhf), using a qemu-debootstrap environment based on https://github.com/multiarch/debian-debootstrap. All binaries in the image are armhf and are emulated on the fly through qemu-user-static.

The AppImageBuild image has been added in AppImageCommunity/AppImageBuild@365a034.

Closes #111.

[TheAssassin]

@probonopd I'm not sure why the built runtime can't be run -- see https://travis-ci.org/AppImage/AppImageKit/jobs/352012266#L3766.

[TheAssassin]

Found the issue. The "insertion" of the magic bytes breaks the runtime file, so that it no longer runs.

I am not 100% sure why, but it's probably related to "unsupported ioctl" error messages I get during all the dd commands.

As a workaround, we could simply insert the bytes outside the Docker container, but it's not an ideal solution, and I would not recommend it.

Ideas welcome.

[TheAssassin]

It's not dd's fault, as far as I can tell. I tried with a Python substitute I wrote (python -c "f=open('runtime', 'r+b'); f.seek(8), f.write('\x41\x49\x02')"), and I get the same errors.

I'll introduce a test into build-runtime.sh calling ./runtime --appimage-version to see whether the runtime runs at all.

[TheAssassin]

Just tried on Armbian, the armhf build works fine. Now, we just need to find out how we can properly embed the magic bytes without breaking the ELF header. Otherwise, the release would violate the spec.

I can embed the magic bytes on my Armbian test system without any problems, therefore the problems are probably caused by qemu-arm-static emulating dd...

[TheAssassin]

So, after some more testing, it seems like the insertion of the magic bytes only breaks the execution with qemu-arm-static, but works fine when running in a "normal" non-emulated environment, e.g., my Armbian, Raspbian, etc. systems I have on hand.

The question is, should we use the workaround I suggested (that is, insert the magic bytes outside the Docker container, e.g., right before uploading the artifacts (although in that case, we'd have to insert them into the embedded object inside appimagetool.AppImage...), or rather have qemu be a little more permissive with ELF files.

This is not a priority change, but we should decide what to do.

[probonopd]

The question is, should we use the workaround I suggested (that is, insert the magic bytes outside the Docker container

Would be fine for me; please be sure to leave comments in the appropriate places as to why we are doing it this way.

[TheAssassin]

@probonopd I have no clue how we could safely alter the runtime embedded in appimagetool, though.

[probonopd]

Uh, that's a bummer...

[TheAssassin]

@probonopd we definitely need to re-think the magic bytes thing for the next AppImage specification. Apparently, the runtime is a valid ELF file until the magic bytes are inserted.

[TheAssassin]

Re. workaround: thinking about it, it doesn't make sense to implement the workaround I suggested, even if we had implemented a method. The problem is that we need to run the tests on Travis. We could use the AppImages built without the magic bytes, sure, but that'll increase the complexity (well, it'd probably be just a few lines of code that run after all the other steps), as we'd need to take care of the insertion as a final step. That breaks our current CI build workflow, that is, build project, run unit tests, build AppImages of tools, test these AppImages, then upload to GitHub.

[TheAssassin]

@probonopd remember the new CMake flag to disable the embedding of magic bytes into the runtime? Might come in handy here as well. We just need to add a comment to make sure people won't misunderstand the intention behind.

[probonopd]

Might come in handy here as well.

Yes, in fact I had assumed that this ticket was the reason for that flag in the first place ;-)

[TheAssassin]

We just need to add a comment to make sure people won't misunderstand the intention behind.

I recently merged #731, which hides the variable from CMake interfaces. It can still be set, but it won't show up in CMake GUI, etc.

[TheAssassin]

I just had another idea how we could work around the execution issues with qemu-arm-static: We could just not embed a runtime at all. It's fairly easy to insert the magic bytes after the build in the container, but it is way harder to embed them into a file embedded in appimagetool. By not embedding the runtime and shipping it separately, we avoid any complex workarounds.

[probonopd]

And then? ARM builds would not contain a runtime and hence need a runtime to be installed on the system? Or would the runtime be inserted at a later time in the process?

[TheAssassin]

@probonopd like on macOS, you'd have to download the runtime and specify it via a CLI parameter.

[probonopd]

@probonopd like on macOS, you'd have to download the runtime and specify it via a CLI parameter.

I never had to download a runtime to run an .app on the Mac?

[TheAssassin]

@probonopd you got this wrong. You remember @teras's changes that allowed building appimagetool on macOS? At that time, we introduced the possibility to use a user-provided runtime when building an AppImage. You don't have to download a runtime to run the AppImage on a Linux computer. You need it on macOS and pass it to appimagetool, though. We could implement the same workflow for ARM platforms. We don't embed the runtime in appimagetool, we require the user to download it separately.

[probonopd]

Ah, right. That's of course a valid way to do it.

[bebuch]

What is the current state of this pull request? Having ARM binaries would be very useful for our CI process.

[probonopd]

Agree, this would be great to have. @TheAssassin what do we need to do to make it happen?

[TheAssassin]

Thinking about this now, we could've simply copied the binaries we just built into a tempdir, remove the magic bytes, and perform our tests. That way, we could circumvent the qemu restrictions. We could do that for all platforms even.

Will re-think about this.

[TheAssassin]

Fixed the remaining issues. The ARM64 AppImage of appimagetool works like a charm. The ARM config build times are roughly equivalent to the "normal" ones, which is great IMO.

I'd appreciate if anyone could try the armhf AppImage, then we can merge the PR.

[probonopd]

Congratulations @TheAssassin the armhf one seems to work like a charm:

root@vegas805:~# ./appimagetool-armhf.AppImage 
appimagetool, continuous build (commit 63ba9c5), build 1978 built on 2018-12-05 00:13:41 UTC
WARNING: appstreamcli command is missing, please install it if you want to use AppStream metadata
SOURCE is missing

root@vegas805:~# ./appimagetool-armhf.AppImage --appimage-extract
squashfs-root/.DirIcon
(...)

root@vegas805:~# ./appimagetool-armhf.AppImage squashfs-root/
appimagetool, continuous build (commit 63ba9c5), build 1978 built on 2018-12-05 00:13:41 UTC
WARNING: appstreamcli command is missing, please install it if you want to use AppStream metadata
Using architecture ARM
/root/squashfs-root should be packaged as appimagetool-ARM.AppImage
AppStream upstream metadata found in usr/share/metainfo/appimagetool.appdata.xml
Generating squashfs...
Parallel mksquashfs: Using 4 processors
Creating 4.0 filesystem on appimagetool-ARM.AppImage, block size 131072.
(...)
Embedding ELF...
Marking the AppImage as executable...
Embedding MD5 digest
Success

Please consider submitting your AppImage to AppImageHub, the crowd-sourced
central directory of available AppImages, by opening a pull request
at https://github.com/AppImage/appimage.github.io

appimagetool-ARM.AppImage: ELF 32-bit LSB executable, ARM, EABI5 version 1, dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=c1b69f457eb838e21c2e8e6b8390f3ddde4fb50e, stripped

root@vegas805:~# uname -a
Linux vegas805 3.10.108 #6 SMP PREEMPT Sun Oct 28 09:34:09 CET 2018 armv7l armv7l armv7l GNU/Linux

root@vegas805:~# cat /etc/os-release 
NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"

root@vegas805:~# dpkg --print-architecture
armhf

root@vegas805:~# cat /proc/cpuinfo 
Processor	: ARMv7 Processor rev 1 (v7l)
processor	: 0
BogoMIPS	: 1.27

processor	: 1
BogoMIPS	: 1.27

processor	: 2
BogoMIPS	: 1.27

processor	: 3
BogoMIPS	: 1.27

Features	: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xc05
CPU revision	: 1

Hardware	: Amlogic Meson8B
Revision	: 000a

But hey, what happened to the AppImage magic bytes?

root@vegas805:~# xxd appimagetool-ARM.AppImage
00000000: 7f45 4c46 0101 0100 5c78 3400 0000 0000  .ELF....\x4.....
00000010: 0200 2800 0100 0000 19c2 0000 3400 0000  ..(.........4...
(...)

Another, minor thing I noticed is that when building an AppImage, the architecture gets named "ARM" rather than "armel":

Using architecture ARM
/root/squashfs-root should be packaged as appimagetool-ARM.AppImage

Is that intentional?

[probonopd]

The 64-bit one says:

root@aml:~# xxd appimagetool-ARM_aarch64.AppImage | head -n 2
00000000: 7f45 4c46 0201 0100 5c78 3400 0000 0000  .ELF....\x4.....
00000010: 0200 b700 0100 0000 8056 4000 0000 0000  .........V@.....

root@aml:~# ./appimagetool-aarch64.AppImage squashfs-root/
(...)
Using architecture ARM_aarch64
/root/squashfs-root should be packaged as appimagetool-ARM_aarch64.AppImage

Is ARM_aarch64 correct? Never read that term anywhere.

root@aml:~# file appimagetool-ARM_aarch64.AppImage 
appimagetool-ARM_aarch64.AppImage: ELF 64-bit LSB executable, ARM aarch64, version 1, dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=56e48d8de4b171bf49dce75862b4a0b579688080, stripped

root@aml:~# uname -a
Linux aml 4.18.7-aml-s9xxx #43 SMP PREEMPT Fri Sep 28 15:09:10 MSK 2018 aarch64 aarch64 aarch64 GNU/Linux

root@aml:~# dpkg --print-architecture
arm64

root@aml:~# cat /proc/cpuinfo 
processor	: 0
model name	: ARMv8 Processor rev 4 (v8l)
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 1
model name	: ARMv8 Processor rev 4 (v8l)
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 2
model name	: ARMv8 Processor rev 4 (v8l)
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 3
model name	: ARMv8 Processor rev 4 (v8l)
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

[TheAssassin]

Should be aarch64 and arm{hf,el} IMO. CC @azubieta you recently worked on that code, any ideas how to fix?

I'll have a look at the magic bytes issue. Strange, though... I thought we had some test in the scripts that should ensure they're embedded properly...

[TheAssassin]

Another occurrence of printf in the runtime build CMake script, taken from the original shell scripts. This command seems to be broken on all ARM shells, not sure why. I tried with echo -en, and it does the job properly during the non-ARM builds, but fails (albeit differently) for ARM.

I added a check to see whether magic bytes are embedded in d48de6d (not bullet proof, but it'll catch major issues like this one), which makes the build fail if the embedding fails, as it should've been before. See https://travis-ci.org/AppImage/AppImageKit/jobs/463758810#L3708 for instance. Trying to find out why this isn't working/what would be working on ARM.

[TheAssassin]

So, echo worked in a shell, but not from CMake. Workaround: actually use a shell. I wrote a tiny bash script to perform the embedding, put it in the source dir next to the runtime build script, et voilà, works.

@probonopd please re-check.

(That script might come in handy in other places, too, by the way...)

[probonopd]

# 32-bit

root@vegas805:~# ./appimagetool-armhf.AppImage squashfs-root/
appimagetool, continuous build (commit 6b66441), build 1990 built on 2018-12-05 11:42:42 UTC
WARNING: appstreamcli command is missing, please install it if you want to use AppStream metadata
Using architecture ARM
/root/squashfs-root should be packaged as appimagetool-ARM.AppImage

root@vegas805:~# xxd appimagetool-ARM.AppImage | head -n 2
00000000: 7f45 4c46 0101 0100 4149 0200 0000 0000  .ELF....AI......
00000010: 0200 2800 0100 0000 19c2 0000 3400 0000  ..(.........4...

The magic bytes are OK but the architecture should be armel rather than ARM (I guess)

# 64-bit

root@aml:~# ./appimagetool-aarch64.AppImage squashfs-root/
appimagetool, continuous build (commit 6b66441), build 1990 built on 2018-12-05 11:43:53 UTC
WARNING: appstreamcli command is missing, please install it if you want to use AppStream metadata
(...)
/root/squashfs-root should be packaged as appimagetool-ARM_aarch64.AppImage

root@aml:~# xxd appimagetool-ARM_aarch64.AppImage | head -n 2
00000000: 7f45 4c46 0201 0100 4149 0200 0000 0000  .ELF....AI......
00000010: 0200 b700 0100 0000 8056 4000 0000 0000  .........V@.....

The magic bytes are OK but the architecture should be aarch64 rather than ARM_aarch64 (I guess)

[TheAssassin]

I fully agree, but I'd say that's a minor issue. We can merge the PR now and fix that eventually. I'll create an issue for that.

[probonopd]

See #886