Cross compiling for Raspberry Pi on Windows

[andre2007]

I try to cross compile to Raspberry Pi from Windows with almost no knowledge on LLVM/CMake/GCC..

Steps done:

• I use ldc2-1.4.0-beta1-win32-msvc
• Zip file ldc-1.4.0-beta1-src is downloaded and extracted (--ldcSrcDir switch)
• ninja / cmake are installed and in PATH environment variable
• GCC toolchain installed from here http://gnutoolchains.com/raspberry/ (2016-09-23-raspbian-jessie)

I execute following comands
set CC=arm-linux-gnueabihf-gcc
ldc-build-runtime --ldcSrcDir="ldc-1.4.0-beta1-src" --ninja

Please find attached the error log. If I understand it correctly, CMake tries to create a windows executable for test purposes. This seems rather strange and cannot work as the cross compiler is intended to create linux arm executables..

log.txt

[kinke]

Joakim has just gotten it to work to cross-compile from Wine to ARM using the Android NDK for Windows, see #2301. You'll essentially have to customize the flags he used to your toolchain (the NDK uses clang, you use gcc...). Appending the CMake vars TARGET_SYSTEM=Linux;UNIX CMAKE_C_COMPILER_WORKS=True CMAKE_SYSTEM_NAME=Linux to the command line will probably get you a bit further, but you'll have to dive into the gcc compiler flags yourself, don't expect a step-by-step guide. ;)

[kinke]

You'll definitely also need --dFlags=-w;-mtriple=arm-linux-gnueabihf in your ldc-build-runtime command-line, so that LDC emits objects for Linux/ARM instead of native Win32 ones.

[andre2007]

Thanks a lot. With these commands:

set CC=arm-linux-gnueabihf-gcc
ldc-build-runtime --ldcSrcDir "ldc-1.4.0-beta1-src" --ninja --dFlags=-w;-mtriple=arm-linux-gnueabihf TARGET_SYSTEM=Linux;UNIX CMAKE_C_COMPILER_WORKS=True CMAKE_SYSTEM_NAME=Linux

I get this output:

CMake Error at CMakeLists.txt:450 (add_library):
  The install of the druntime-ldc-shared target requires changing an RPATH
  from the build tree, but this is not supported with the Ninja generator
  unless on an ELF-based platform.  The CMAKE_BUILD_WITH_INSTALL_RPATH
  variable may be set to avoid this relinking step.

I will try further whether I can solve this issue using the mentioned CMAKE_BUILD_WITH_INSTALL_RPATH variable.

[kinke]

You can append BUILD_SHARED_LIBS=OFF to skip the generation of the shared libraries.

[andre2007]

Almost there:)
It now fails because it cannot find unwind.h

Invoking: ["ninja", "all"]
[5/50] Building C object CMakeFiles/druntime-ldc-debug.dir/druntime/src/ldc/arm_unwind.c.o
FAILED: CMakeFiles/druntime-ldc-debug.dir/druntime/src/ldc/arm_unwind.c.o
C:\SysGCC\Raspberry\bin\arm-linux-gnueabihf-gcc.exe   -fPIC -MD -MT CMakeFiles/druntime-ldc-debug.dir/druntime/src/ldc/arm_unwind.c.o -MF CMakeFiles\druntime-ldc-debug.dir\druntime\src\ldc\arm_unwind.c.o.d -o CMakeFiles/druntime-ldc-debug.dir/druntime/src/ldc/arm_unwind.c.o   -c C:/D/ldc2-1.4.0-beta1-win32-msvc/bin/ldc-1.4.0-beta1-src/runtime/druntime/src/ldc/arm_unwind.c
C:/D/ldc2-1.4.0-beta1-win32-msvc/bin/ldc-1.4.0-beta1-src/runtime/druntime/src/ldc/arm_unwind.c:15:20: fatal error: unwind.h: No such file or directory
 #include <unwind.h>
                    ^
compilation terminated.

I checked the tool chain. In the folder gcc\arm-linux-gnueabihf\4.9\include
there is no file "unwind.h" but only "unwind-arm-common.h"

There is another folder "arm-linux-gnueabihf\sysroot\usr\lib\gcc\arm-linux-gnueabihf\4.9.2\include" which contains "unwind.h" and "unwind-arm-common.h". I will try to find out whether this is an issue with the toolchain..

[andre2007]

Solved, I copied unwind.h from the folder where it exists to the other folder where it was missing.
Runtime libraries built:)

[kinke]

Great. :) Could you please specify your final command-line?

[andre2007]

Yes:)

set CC=arm-linux-gnueabihf-gcc
ldc-build-runtime --ldcSrcDir "ldc-1.4.0-beta1-src" --ninja --dFlags=-w;-mtriple=arm-linux-gnueabihf TARGET_SYSTEM=Linux;UNIX CMAKE_C_COMPILER_WORKS=True CMAKE_SYSTEM_NAME=Linux BUILD_SHARED_LIBS=OFF

I created a post on the forum of the gcc toolchain site
https://sysprogs.com/w/forums/topic/unwind-h-is-missing-pi-toolchain-for-windows/

[kinke]

Alright, that's how short I hoped the command lines to be when using a cross-gcc. Can you try adding --testrunners? That'll test whether linking works too, and if so, you can then run the unittest executables on your Pi.

[andre2007]

yes I will try. Maybe a question, how do I now create an executable from e.g. sieve.d?
By looking at the example from here https://wiki.dlang.org/Build_D_for_Android
I thought I compile it with command:

ldc2 -c sieve.d
arm-linux-gnueabihf-gcc sieve.obj -o executable

ldc2 creates an obj file. From the wiki example it seems a o file is created...

sieve.obj: file not recognized: File format not recognized
collect2.exe: error: ld returned 1 exit status

[andre2007]

The executable was created fine, just a few warnings. I have to copy it to my raspberry pi and will tell you the result

Invoking: ["ninja", "all", "all-test-runners"]
[29/880] Generating objects-unittest/std/math.o
ieeeFlags test disabled, see LDC Issue #888
test disabled on LDC, see bug 5628
[38/880] Generating objects-debug/etc/c/curl.o, objects-debug/etc/c/odbc/sql.o, objects-deb...std/c/linux/socket.o, objects-debug/std/c/linux/termios.o, objects-debug/std/c/linux/tipc.ostd\c\linux\pthread.d(17): Deprecation: module std.c.linux.linux is deprecated - Import the appropriate core.sys.posix.* modules instead
[48/880] Generating objects-unittest/std/experimental/typecons.o
Could not locate matching function for: FuncInfo!("foo", void())
Could not locate matching function for: FuncInfo!("bar", void())
Could not locate matching function for: FuncInfo!("foo", void())
Could not locate matching function for: FuncInfo!("refresh", int())
Could not locate matching function for: FuncInfo!("refresh", int())
Could not locate matching function for: FuncInfo!("refresh", int())
[71/880] Generating objects-unittest/std/c/linux/pthread.o
std\c\linux\pthread.d(17): Deprecation: module std.c.linux.linux is deprecated - Import the appropriate core.sys.posix.* modules instead
[80/880] Generating objects-unittest/std/traits.o
std\traits.d-mixin-7672(7672): Deprecation: std.internal.test.uda.HasPrivateMembers.c is not visible from module traits
std\traits.d-mixin-7672(7672): Deprecation: std.internal.test.uda.HasPrivateMembers.d is not visible from module traits
std\traits.d-mixin-7664(7664): Deprecation: std.internal.test.uda.HasPrivateMembers.c is not visible from module traits
[97/880] Generating objects-unittest/std/uri.o
std\uri.d(323): Deprecation: function std.utf.toUTF8 is deprecated - To be removed November 2017. Please use std.utf.encode instead.
std\uri.d(323): Deprecation: function std.utf.toUTF8 is deprecated - To be removed November 2017. Please use std.utf.encode instead.
std\uri.d(323): Deprecation: function std.utf.toUTF8 is deprecated - To be removed November 2017. Please use std.utf.encode instead.
[173/880] Generating objects-unittest-debug/rt/util/typeinfo.o
rt\util\typeinfo.d(196): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(196): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(196): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(244): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(244): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(244): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(298): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(298): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(298): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
[203/880] Generating objects/etc/c/curl.o, objects/etc/c/odbc/sql.o, objects/etc/c/odbc/sql....o, objects/std/c/linux/socket.o, objects/std/c/linux/termios.o, objects/std/c/linux/tipc.ostd\c\linux\pthread.d(17): Deprecation: module std.c.linux.linux is deprecated - Import the appropriate core.sys.posix.* modules instead
[353/880] Generating objects-unittest-debug/core/sync/mutex.o
core\sync\mutex.d(319): Deprecation: read-modify-write operations are not allowed for shared variables. Use core.atomic.atomicOp!"+="(this.cargo, 1) instead.
[556/880] Generating objects-unittest/core/sync/mutex.o
core\sync\mutex.d(319): Deprecation: read-modify-write operations are not allowed for shared variables. Use core.atomic.atomicOp!"+="(this.cargo, 1) instead.
[626/880] Generating objects-unittest/rt/util/typeinfo.o
rt\util\typeinfo.d(196): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(196): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(196): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(244): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(244): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(244): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(298): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(298): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
rt\util\typeinfo.d(298): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
[781/880] Generating objects-unittest-debug/std/experimental/typecons.o
Could not locate matching function for: FuncInfo!("foo", void())
Could not locate matching function for: FuncInfo!("bar", void())
Could not locate matching function for: FuncInfo!("foo", void())
Could not locate matching function for: FuncInfo!("refresh", int())
Could not locate matching function for: FuncInfo!("refresh", int())
Could not locate matching function for: FuncInfo!("refresh", int())
[804/880] Generating objects-unittest-debug/std/math.o
ieeeFlags test disabled, see LDC Issue #888
test disabled on LDC, see bug 5628
[835/880] Generating objects-unittest-debug/std/uri.o
std\uri.d(323): Deprecation: function std.utf.toUTF8 is deprecated - To be removed November 2017. Please use std.utf.encode instead.
std\uri.d(323): Deprecation: function std.utf.toUTF8 is deprecated - To be removed November 2017. Please use std.utf.encode instead.
std\uri.d(323): Deprecation: function std.utf.toUTF8 is deprecated - To be removed November 2017. Please use std.utf.encode instead.
[841/880] Generating objects-unittest-debug/std/traits.o
std\traits.d-mixin-7672(7672): Deprecation: std.internal.test.uda.HasPrivateMembers.c is not visible from module traits
std\traits.d-mixin-7672(7672): Deprecation: std.internal.test.uda.HasPrivateMembers.d is not visible from module traits
std\traits.d-mixin-7664(7664): Deprecation: std.internal.test.uda.HasPrivateMembers.c is not visible from module traits
[851/880] Generating objects-unittest-debug/std/c/linux/pthread.o
std\c\linux\pthread.d(17): Deprecation: module std.c.linux.linux is deprecated - Import the appropriate core.sys.posix.* modules instead
[880/880] Linking C executable phobos2-test-runner-debug
Runtime libraries built successfully into: C:\D\ldc2-1.4.0-beta1-win32-msvc\bin\ldc-build-runtime.tmp

[kinke]

how do I now create an executable from e.g. sieve.d?
[...]
ldc2 -c sieve.d
arm-linux-gnueabihf-gcc sieve.obj -o executable
[...]
sieve.obj: file not recognized: File format not recognized

Firstly, it should be clear by now that you need to tell LDC to cross-compile to ARM via -mtriple=arm-linux-gnueabihf, it obviously targets native Win32 by default.
Then for linking, you need to specify the cross-compiled runtime libs: gcc -L.../ldc-build-runtime.tmp/lib -ldruntime-ldc -lphobos2-ldc. The other option is to link via LDC, telling it about your cross-gcc and runtime libs directory: ldc2 -mtriple=arm-linux-gnueabihf helloWorld.d -gcc=.../gcc.exe -L-L.../ldc-build-runtime.tmp/lib.

[andre2007]

It seems there are some issues with the testrunner on raspberry pi.
I started the application with command: sudo ./phobos2-test-runner-debug &> log2.txt
I run it for 5 hours. After core.bitop it stops.
I run it a second time with the same result. Please find log2.txt attached.
log2.txt

If I run it with command sudo ./phobos2-test-runner-debug
some more tests are executed but it also hangs. This time on gc.bits.
Also this is reproducable. See log3.txt
log3.txt

[kinke]

There's apparently at least a threading issue with the pthreads library. You may want to try debugging a bit:

gdb --args ./phobos2-test-runner-debug std.stdio
r
bt

[andre2007]

Calling the command for std.stdio suceeds. It is strange because if all tests are called there is an error in log (log3.txt).

I also called the command for core.thread. Here the result:

(gdb) r
Starting program: /home/pi/phobos2-test-runner-debug core.thread
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
[New Thread 0x76c67630 (LWP 17530)]
[Thread 0x76c67630 (LWP 17530) exited]
[New Thread 0x762fc630 (LWP 17531)]
[New Thread 0x75afc630 (LWP 17532)]

Program received signal SIGUSR1, User defined signal 1.
0x76f0b274 in pthread_join (threadid=<optimized out>, thread_return=0x0) at pthread_join.c:92
92      pthread_join.c: No such file or directory.
(gdb) bt
#0  0x76f0b274 in pthread_join (threadid=<optimized out>, thread_return=0x0) at pthread_join.c:92
#1  0x04a16bc0 in core.thread.Thread.join() (this=0x76c6b200, rethrow=true) at thread.d:772
#2  0x04a17d54 in core.thread.Thread.__unittestL1253_3() () at thread.d:1263
#3  0x04a20ba4 in core.thread.__unittest() ()
#4  0x02f346e0 in test_runner.doTest() (moduleInfo=0x4ecec40 <core.thread.__ModuleInfo()>, ret=<optimized out>) at test_runner.d:56
#5  0x02f345fc in test_runner.testModules() () at test_runner.d:30
#6  0x02f343e0 in test_runner.tester() () at test_runner.d:13
#7  0x04a0f188 in runModuleUnitTests () at runtime.d:522
#8  0x04ad3d9c in rt.dmain2._d_run_main() () at dmain2.d:477
#9  0x04ad3ce8 in rt.dmain2._d_run_main() (dg=...) at dmain2.d:453
#10 0x04ad3be8 in _d_run_main (argc=2, argv=0x7efff344, mainFunc=0x2f34a28 <D main>) at dmain2.d:486
#11 0x02f34bac in main (argc=2, argv=0x7efff344) at __entrypoint.d:8

Maybe it is related to the missing unwind.h file which was missing and I needed to copy from another folder?

[dnadlinger]

Wasn't SIGUSR1 in use by the Android runtime somehow? https://github.com/ldc-developers/druntime/blob/4d3b8c17936089f542ddaee00db9abc34e8e260a/src/core/thread.d#L1975-L1978

[dnadlinger]

(I.e. this might be expected, and you might need to handle nostop SIGUSR1 or manually continue a few times to advance the tests.)

[andre2007]

I also now tried to run the phobos unittest executable which does not contain debug information. The result is quite different. A segmentation fault is occuring

****** FAIL release32 std.random
core.exception.AssertError@std\random.d(3092): Assertion failure
----------------
./phobos2-test-runner(_D4core7runtime18runModuleUnitTestsUZ19unittestSegvHandlerUNbiPS4core3sys5posix6signal9siginfo_tPvZv+0x28)[0x43c63c4]
/lib/arm-linux-gnueabihf/libc.so.6(__default_rt_sa_restorer_v2+0x0)[0x76c9b1a0]
./phobos2-test-runner(_D11test_runner6doTestFPS6object10ModuleInfoKbZv+0x14c)[0x2df451c]

log-no-debug.txt

The runtime unittests are running just fine, except a warning:
0.004s PASS release32 core.sync.semaphore
Not safe to migrate Fibers between Threads on your system. Consider setting version CheckFiberMigration for this system in thread.d
0.539s PASS release32 core.thread

[andre2007]

I just noticed that Raspberry Pi is a 64 Bit ARM system. As this is work in progress (#2153) maybe some issues are related to this.

I also tried to add the cflag and linkerflag -pthread but that solved no of the issues.

[kinke]

But you're not targeting AArch64 with -mtriple=arm-linux-gnueabihf, that's 32-bit ARM.

[andre2007]

Just ignore what I have said, I just validated that my raspberry pi 2 is 32 Bit. The new raspberry pi 3 is 64 Bit.

[joakim-noah]

For the std.random failure, try commenting out this assert. As for the hangs, not important as those are the druntime tests, which are only run after the Phobos tests, seen that on linux/x64 too.

[PetarKirov]

@andre2007 note that even with RPi3 chances are that you would be using 32-bit ARMv6, because that's only thing that Raspbian supports. 64-bit support only recently started to emerge from SUSE.

As to RPi3, I was able to compile LDC 1.1 natively on RPi3 and then use it to compile a small vibe.d application back in December last year without issues IIRC, though I haven't time to work with D on RPi since then.

[kinke]

Is there anything left to do here? The std.random issue is #2024 (comment).

[andre2007]

You are right, this issue can be closed.