Native binaries for ARMv6 (Pi Zeros) are incompatible.

[savageautomate]

As noted in issue #27, the native builds for ARMv6 (Pi Zeros and all Generation 1 models) are not working.
The 32-bit binaries compiled are not ARMv6 compatible.

This issue affects the following Raspberry Pi models

• RPi 1 Model A
• RPi 1 Model A+
• RPi 1 Model B
• RPi 1 Model B+
• RPi Compute Module 1
• RPi Zero
• RPi Zero W

(from https://en.wikipedia.org/wiki/Raspberry_Pi)

Changes to native build happened here: a7c29d0

I switched from ARM compiler toolchain available from RaspberryPi Tools to a newer GCC version 32-bit cross-compiler (gcc-arm-linux-gnueabihf) toolchain available in APT repositories.

REF: https://github.com/Pro/raspi-toolchain

By default, newer GCC versions do not create correct binaries for ARMv6. Even though you pass the correct -mcpu= flag to gcc, it will create startup code for the newer ARMv7 architecture. Running them on your RasPI Zero will cause an "Illegal Instruction" exception.

I probably need to instrument one of these custom toolchains in the build logic for building 32-bit rather than the default linarogcc-arm-linux-gnueabihf

[hackerjimbo]

For reference, the correct flags to gcc seem to be -march=armv6+fp -mfloat-abi=hard -mfpu=vfp -marm. That final -marm seems to be because by default the compiler emits thumb mode code and that doesn't play well with VFP on the v6 architecture.

[hackerjimbo]

Investigating further it appears that while gcc is correctly build v6 code, the support libraries are built with the default v7 code. This would mean that the .o files are OK but some some reason v7 code gets introduced when the .so is created.

If we're linking in pigpio at the same time (are we?) then that also needs to be v6 compiled.

[savageautomate]

Investigating further it appears that while gcc is correctly build v6 code, the support libraries are built with the default v7 code. This would mean that the .o files are OK but some some reason v7 code gets introduced when the .so is created.

If we're linking in pigpio at the same time (are we?) then that also needs to be v6 compiled.

Yes we link against dynamic library libpigpio.so. I don't believe there is any static linking. We do compile libpigpio.so as part of the build ... but only if it's not detected/previously compiled, so I bet that part of the build also needs these flags.

You would probably needs to add CFLAGS=$CARGS here as well to build PIGPIO with the same options:
https://github.com/Pi4J/pi4j-v2/blob/75e1909006ef8fae8ad02d44f48096b368c634fd/libraries/pi4j-library-pigpio/src/main/native/build-libpigpio.sh#L62-L69

You should be able to run a maven clean which would force the PIGPIO library to get rebuilt.

mvn clean install -Pnative

If all this works by just adding the needed compiler flags, that would be terrific and a very simple fix!

Thanks, Robert

[savageautomate]

I started testing these flags on my side and got compile errors.

I think the reason it compiles with the flags you suggested (-march=armv6+fp -mfloat-abi=hard -mfpu=vfp -marm) for you is that you are compiling directly on a Raspberry Pi. When I compile with the Docker image which uses ARM cross-compiler toolchains pulled from the APT repository from Ubuntu, I get this:

[INFO]      [exec] arm-linux-gnueabihf-gcc: note: valid arguments to '-march=' are: armv2 armv2a armv3 armv3m armv4 armv4t armv5 armv5e armv5t armv5te armv5tej armv6 armv6-m armv6j armv6k armv6kz armv6s-m armv6t2 armv6z armv6zk armv7 armv7-a armv7-m armv7-r armv7e-m armv7ve armv8-a armv8-a+crc armv8-m.base armv8-m.main armv8-m.main+dsp armv8.1-a armv8.2-a armv8.2-a+dotprod armv8.2-a+fp16 armv8.2-a+fp16+dotprod iwmmxt iwmmxt2 native; did you mean 'armv6'?
[INFO]      [exec] make: *** [com_pi4j_library_pigpio_internal_PIGPIO.o] Error 1

So, I think I'm still going to have to investigate alternative toolchains for the build.

[savageautomate]

I was able to get these compiler flags working. "-march=armv6 -mfloat-abi=hard -mfpu=vfp -marm"

You can test it from this branch. if you like:
https://github.com/Pi4J/pi4j-v2/tree/issue/%2328

I have not tested on hardware. I don't have the hardware readily accessible at the moment.

[hackerjimbo]

I've tried that branch and it doesn't work. The issue is in the function register_tm_clones which is part of the C++ runtime support that appears to be burnt into the .so file when its' created. The compiler is just using the system one which, for 32-bit arm, assume v7.

Some history here. Debian, from whence we get the compile, does not support ARM v6 especially with hard FP. This is why the very first Pi OS was debian with soft FP as that used to be supported. Raspbian was big undertaking to rebuild the universe based on a v6 architecture with hardware floating point. Because the cross-compiler's libraries assume v7 is all goes wrong.

So, it looks like the only way this'll work is to use the cross compiler in docker that was referenced from wherever it was.

[savageautomate]

The GCC compiler is pretty old in the RaspberryPi Tools repo that I was originally using.
So maybe first we can try the one published here and see if they work for ARMv6+FP.

I'll try to get this working soon.

[savageautomate]

@hackerjimbo

Well, been struggling with this all day. I believe I have it working using the original RaspberryPi Tools cross compiler toolchains when building with Docker ... but only on x86_64 machines. This cross-compiler toolchain does not support running on ARM. I also have not found a toolchain to run on aarch64 that will compile for ARMv6.

Do you have an x86_64 machine you can build from to test the ARMv6 compatibility?

mvn clean install -Pnative,docker

Thanks, Robert

[hackerjimbo]

@savageautomate

Is there a specific reason you used the Raspberry Pi Tools rather than the other cross-compiler at https://github.com/Pro/raspi-toolchain ? Also, I'll have a go with both on an aarch64 to see what happens. I'm surprised the RaspberryPi Tools one didn't work on aarch64.

How does the tool chain know which docker image to use and how to power it? I'll have to investigate!

OK, just figured it out. I'll have a play and get back to you. I'd like to be able to have the while thing Pi-hosted, even if it means insisting on the 64-bit OS. It perhaps would be nice to be able to build on 32-bit in the knowledge that it could only make the 32-bit version.

[savageautomate]

@hackerjimbo ,

Is there a specific reason you used the Raspberry Pi Tools rather than the other cross-compiler at https://github.com/Pro/raspi-toolchain ?

Well, I really just wanted to get it working again with a known working compiler and the try to update it to a newer version. I did look into this one and it does not provide a armhf (armv6) cross-compiler for the aarch64 platform. At least not as far as I could tell reading the documentation.

Also, I'll have a go with both on an aarch64 to see what happens. I'm surprised the RaspberryPi Tools one didn't work on aarch64.

The RaspberryPi Tools seems to only include build tools compiled for x86 and x64 binaries. So the trick will be to figure out a way to build for ARMv6 on the aarch64 platform. That's where I'm stuck at the moment. I'm sure we could build the toolchain ourselves, but that is a bit more than I was trying to take on at the moment.

How does the tool chain know which docker image to use and how to power it? I'll have to investigate!

Well, I publish Docker images for both linux/amd64 and linux/arm64 platforms. These Docker images were built installing the cross-compiling toolchains gcc-aarch64-linux-gnu and gcc-arm-linux-gnueabihf packages using apt-get from their respective repositories. When your local Docker attempts to get the Pi4J Docker Builder image from the DockerHub repo, it will pull the correct one based on the host's architecture. (some Docker magic)

Our Maven build scripts support the -pdocker profile which will call the build-docker.sh script in the native project sources. The docker profile is enabled by default on Windows and MacOS, but not currently on Linux systems.

I'd like to be able to have the while thing Pi-hosted, even if it means insisting on the 64-bit OS.

Me too.

It perhaps would be nice to be able to build on 32-bit in the knowledge that it could only make the 32-bit version.

We could with some refactoring of the build logic. At the moment we could build for armhf on a 32-bit Pi OS and aarch64 on a 64-bit Pi OS. The trouble comes from trying to build both and satisfy all the build artifacts that are intended to get embedded in the JAR file.