New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ARMv6 / ARMv7 support #3424

Merged
merged 10 commits into from Oct 28, 2016

Conversation

Projects
None yet
@ysbaddaden
Member

ysbaddaden commented Oct 15, 2016

The ABI is a rework from the work @ssvb did (port from Rust) to which I merely added support for the LLVM Array type.

The arm-unknown-linux-gnueabihf target is for the Raspberry Pi (Raspbian). I only tested in QEMU emulating the ARM1176 CPU, not on a real hardware.

The fiber switch context took quite some time (×2), but seems to be working (now).

I could cross-compile Crystal itself (non release mode), as well as my posix project, which was capable to regenerate the C bindings (in QEMU). I didn't try to build Crystal with Crystal (QEMU is very slow) but Crystal was capable to build simple programs.

The one thing NOT working in unwind. Trying to unwind to generate the stacktrace segfaults, raising an exception fails (phase1 fatal error). Maybe this is because of libgcc_s, which is always linked in, but misses some symbols: _Unwind_GetIP, _Unwind_GetGR and _Unwind_SetGR (despite LSB stating they must be available). Linking against libunwind passes, but maybe it's causing conflicts? See my comment below. Now it works!

@meew0

This comment has been minimized.

Show comment
Hide comment
@meew0

meew0 Oct 16, 2016

I ran into some issues using this PR to cross-compile to a Raspberry Pi:

Firstly, something was missing that caused code using the SecureRandom module to not compile properly. The minimal example

require "secure_random"

puts SecureRandom.urlsafe_base64(18)

led to the following error: https://gist.github.com/meew0/daf2c57b0de3878ae85e8e38bc6b0282

Fortunately, this wasn't a large problem for me, as SecureRandom wasn't actually necessary in my particular code. After replacing it and building again, I ran into the following segfault (here with the compiler -s flag): https://gist.github.com/meew0/07a8a063a31837cfd7fe275d54fa8f56

Note that this only occurred with the --release flag; without it the compilation worked fine. Other than that, thanks a lot for this PR; it came just in time when I needed it!

meew0 commented Oct 16, 2016

I ran into some issues using this PR to cross-compile to a Raspberry Pi:

Firstly, something was missing that caused code using the SecureRandom module to not compile properly. The minimal example

require "secure_random"

puts SecureRandom.urlsafe_base64(18)

led to the following error: https://gist.github.com/meew0/daf2c57b0de3878ae85e8e38bc6b0282

Fortunately, this wasn't a large problem for me, as SecureRandom wasn't actually necessary in my particular code. After replacing it and building again, I ran into the following segfault (here with the compiler -s flag): https://gist.github.com/meew0/07a8a063a31837cfd7fe275d54fa8f56

Note that this only occurred with the --release flag; without it the compilation worked fine. Other than that, thanks a lot for this PR; it came just in time when I needed it!

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 16, 2016

Member

Thanks for testing! There must be some tweaks to do in the stdlib, like you found out for SecureRandom. And certainly some issues to be debugged, too (maybe the fiber context switch is a bit wrong).

How did it crash? You cross compiled the compiler, then tried to build your app? I got a segfault with the compiler built in release mode, too (at the same pass), but my linux has LLVM 3.8 when the QEMU image has LLVM 3.5. We should verify with the exact LLVM version.

You can try to cross compile your app directly, and verify if it passes when compiled with --release, and see how it goes.

Member

ysbaddaden commented Oct 16, 2016

Thanks for testing! There must be some tweaks to do in the stdlib, like you found out for SecureRandom. And certainly some issues to be debugged, too (maybe the fiber context switch is a bit wrong).

How did it crash? You cross compiled the compiler, then tried to build your app? I got a segfault with the compiler built in release mode, too (at the same pass), but my linux has LLVM 3.8 when the QEMU image has LLVM 3.5. We should verify with the exact LLVM version.

You can try to cross compile your app directly, and verify if it passes when compiled with --release, and see how it goes.

@meew0

This comment has been minimized.

Show comment
Hide comment
@meew0

meew0 Oct 16, 2016

Sorry for being unclear there; I compiled the compiler with ARM support on x86 (in release mode), then used that to cross compile my app to ARM (in release/debug mode). Should I try cross-compiling the compiler in debug mode and then using that compiler to compile my app directly on the RasPi?

meew0 commented Oct 16, 2016

Sorry for being unclear there; I compiled the compiler with ARM support on x86 (in release mode), then used that to cross compile my app to ARM (in release/debug mode). Should I try cross-compiling the compiler in debug mode and then using that compiler to compile my app directly on the RasPi?

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 16, 2016

Member

You mean the compiler crashed on x86 while cross-compiling your app, it didn't crash an the ARM board? that's really unexpected 😕

Member

ysbaddaden commented Oct 16, 2016

You mean the compiler crashed on x86 while cross-compiling your app, it didn't crash an the ARM board? that's really unexpected 😕

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 16, 2016

Member

After some more investigating, the Unwind issue is two fold:

  • libgcc_s:
    • is capable to throw exceptions (__crystal_personality is called),
    • is missing symbols (_Unwind_GetIP, _Unwind_SetIP and _Unwind_GetGR), because they're not part of ARM EHABI;
  • libgcc_s, using _Unwind_VRS_{Get,Set}:
    • is capable to unwind the stack EDIT: actually calls __crystal_personality with UA_FORCE_UNWIND and segfaults,
    • calls __crystal_personality but segfaults in _Unwind_GetRegionStart or _Unwind_VRS_Get if stack was unwinded before and always segfaults in _Unwind_GetLanguageSpecificData anyway;
  • libunwind (deb package, v1.1):
    • has all symbols (stubs?),
    • is capable to unwind the stack,
    • is incapable to raise an exception because it can't find an exception handler (with --debug then _Unwind_RaiseException returns END_OF_STACK and __crystal_personality is never called, otherwise ``);
  • llvm/libunwind (master, against LLVM 3.5):
    • has all symbols (stubs),
    • always segfaults in uwn_proc_info.

This kinda looks like a dead end. We could use libunwind to unwind the stack, and libgcc_s to raie the exception ... but we need SetIP and GetGR in the exception handler so it won't work.

Maybe using LLVM's libunwind would help. Maybe using the latest GNU's libunwind could help, too.

EDIT: I found the solution to use libgcc_s: use _Unwind_VRS_Get and _Unwind_VRS_Set. See llvm-mirror/libunwind@2d2bf2f or ARM EHABI for details.

Member

ysbaddaden commented Oct 16, 2016

After some more investigating, the Unwind issue is two fold:

  • libgcc_s:
    • is capable to throw exceptions (__crystal_personality is called),
    • is missing symbols (_Unwind_GetIP, _Unwind_SetIP and _Unwind_GetGR), because they're not part of ARM EHABI;
  • libgcc_s, using _Unwind_VRS_{Get,Set}:
    • is capable to unwind the stack EDIT: actually calls __crystal_personality with UA_FORCE_UNWIND and segfaults,
    • calls __crystal_personality but segfaults in _Unwind_GetRegionStart or _Unwind_VRS_Get if stack was unwinded before and always segfaults in _Unwind_GetLanguageSpecificData anyway;
  • libunwind (deb package, v1.1):
    • has all symbols (stubs?),
    • is capable to unwind the stack,
    • is incapable to raise an exception because it can't find an exception handler (with --debug then _Unwind_RaiseException returns END_OF_STACK and __crystal_personality is never called, otherwise ``);
  • llvm/libunwind (master, against LLVM 3.5):
    • has all symbols (stubs),
    • always segfaults in uwn_proc_info.

This kinda looks like a dead end. We could use libunwind to unwind the stack, and libgcc_s to raie the exception ... but we need SetIP and GetGR in the exception handler so it won't work.

Maybe using LLVM's libunwind would help. Maybe using the latest GNU's libunwind could help, too.

EDIT: I found the solution to use libgcc_s: use _Unwind_VRS_Get and _Unwind_VRS_Set. See llvm-mirror/libunwind@2d2bf2f or ARM EHABI for details.

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 17, 2016

Member

@waj I give up the unwind stuff. This is too complicated for me to fix 😭 Maybe I'll try on Alpine Linux armhf but I think I'll have the same result.

If someone is motivated enough to follow up, here is a patch for using either libunwind or GetGR/SetGR: https://gist.github.com/ysbaddaden/67a2a4ab9c076aa96aa886f5f3d9a06b

Member

ysbaddaden commented Oct 17, 2016

@waj I give up the unwind stuff. This is too complicated for me to fix 😭 Maybe I'll try on Alpine Linux armhf but I think I'll have the same result.

If someone is motivated enough to follow up, here is a patch for using either libunwind or GetGR/SetGR: https://gist.github.com/ysbaddaden/67a2a4ab9c076aa96aa886f5f3d9a06b

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 17, 2016

Member

I understood why __crystal_personality caused segfaults (using libgcc_s). It's being called with UNW_UA_FORCE_UNWIND (_Unwind_Backtrace) or UNW_UA_END_OF_STACK (_Unwind_RaiseException), but calling _Unwind_GetLanguageSpecificData in those cases segfaults because context is invalid.

Prepending the following to __crystal_personality fixes the segfaults:

if actions.end_of_stack?
  return LibUnwind::ReasonCode::END_OF_STACK
end

We end up with the following:

CallStack.unwind()
<collects the backtrace>
__crystal_personality(8, (FORCE_UNWIND, END_OF_STACK), 612757598630632, Pointer(LibUnwind::Exception).null, Pointer(Void).null)

__crystal_raise(Pointer(LibUnwind::Exception)@0x96f30)
__crystal_personality(0, (END_OF_STACK), 612757598630680, Pointer(LibUnwind::Exception)@0xffffffff, Pointer(Void)@0x96f30)
Failed to raise an exception: FAILURE
[244224] *CallStack::print_backtrace:Int32 +-1090522800
[143892] __crystal_raise +-1090522800
[151176] *raise<Exception>:NoReturn +-1090522800
[150888] *raise<String>:NoReturn +-1090522800
[238912] *failure:NoReturn +-1090522800
__crystal_personality(8, (FORCE_UNWIND, END_OF_STACK), 612757598630632, Pointer(LibUnwind::Exception).null, Pointer(Void).null)
[142324] ???
[Inferior 1 (process 5620) exited with code 011]

Note that unwind_ex (at 0x963f0) eventually ends up being the 5th parameter to __crystal_personality when it's supposed to be the 4th ... i.e. there is something fishy.

Member

ysbaddaden commented Oct 17, 2016

I understood why __crystal_personality caused segfaults (using libgcc_s). It's being called with UNW_UA_FORCE_UNWIND (_Unwind_Backtrace) or UNW_UA_END_OF_STACK (_Unwind_RaiseException), but calling _Unwind_GetLanguageSpecificData in those cases segfaults because context is invalid.

Prepending the following to __crystal_personality fixes the segfaults:

if actions.end_of_stack?
  return LibUnwind::ReasonCode::END_OF_STACK
end

We end up with the following:

CallStack.unwind()
<collects the backtrace>
__crystal_personality(8, (FORCE_UNWIND, END_OF_STACK), 612757598630632, Pointer(LibUnwind::Exception).null, Pointer(Void).null)

__crystal_raise(Pointer(LibUnwind::Exception)@0x96f30)
__crystal_personality(0, (END_OF_STACK), 612757598630680, Pointer(LibUnwind::Exception)@0xffffffff, Pointer(Void)@0x96f30)
Failed to raise an exception: FAILURE
[244224] *CallStack::print_backtrace:Int32 +-1090522800
[143892] __crystal_raise +-1090522800
[151176] *raise<Exception>:NoReturn +-1090522800
[150888] *raise<String>:NoReturn +-1090522800
[238912] *failure:NoReturn +-1090522800
__crystal_personality(8, (FORCE_UNWIND, END_OF_STACK), 612757598630632, Pointer(LibUnwind::Exception).null, Pointer(Void).null)
[142324] ???
[Inferior 1 (process 5620) exited with code 011]

Note that unwind_ex (at 0x963f0) eventually ends up being the 5th parameter to __crystal_personality when it's supposed to be the 4th ... i.e. there is something fishy.

@asterite

This comment has been minimized.

Show comment
Hide comment
@asterite

asterite Oct 19, 2016

Contributor

Hey @ysbaddaden, this is really awesome!!

Do you think we can still merge this anyway? In this way it doesn't get outdated and trying to make the exception-handling work is a bit easier (no need to work in a separate branch and constantly rebase against master). What do you think?

Contributor

asterite commented Oct 19, 2016

Hey @ysbaddaden, this is really awesome!!

Do you think we can still merge this anyway? In this way it doesn't get outdated and trying to make the exception-handling work is a bit easier (no need to work in a separate branch and constantly rebase against master). What do you think?

@ssvb

This comment has been minimized.

Show comment
Hide comment
@ssvb

ssvb Oct 19, 2016

@ysbaddaden

The ABI is a rework from the work @ssvb did (port from Rust) to which I merely added support for the LLVM Array type.

I had omitted the LLVM Array type support on purpose, because it did not seem to be used by Crystal in any way. I generally don't feel very comfortable about having untested code branches. Missing this particular functionality is perfectly fine because in the case if Crystal ever needs it, the error message would be very clear and explicit. The rest of the ARM ABI code paths are more or less validated. Some of them even used a minimalistic standalone test program for this as mentioned in #324 (comment)

Thanks a lot for taking care of the fiber context switch code. The ancient Crystal did not need it, so we could get away without this functionality.

Now the remaining bits are the support for raising exceptions (the libunwind stuff) and LLVM code generation problems (enabling/disabling optimizations was a bit fragile). I haven't tested your branch yet, but maybe will try to find some time for this.

ssvb commented Oct 19, 2016

@ysbaddaden

The ABI is a rework from the work @ssvb did (port from Rust) to which I merely added support for the LLVM Array type.

I had omitted the LLVM Array type support on purpose, because it did not seem to be used by Crystal in any way. I generally don't feel very comfortable about having untested code branches. Missing this particular functionality is perfectly fine because in the case if Crystal ever needs it, the error message would be very clear and explicit. The rest of the ARM ABI code paths are more or less validated. Some of them even used a minimalistic standalone test program for this as mentioned in #324 (comment)

Thanks a lot for taking care of the fiber context switch code. The ancient Crystal did not need it, so we could get away without this functionality.

Now the remaining bits are the support for raising exceptions (the libunwind stuff) and LLVM code generation problems (enabling/disabling optimizations was a bit fragile). I haven't tested your branch yet, but maybe will try to find some time for this.

@ssvb

This comment has been minimized.

Show comment
Hide comment
@ssvb

ssvb Oct 19, 2016

@asterite

Do you think we can still merge this anyway? In this way it doesn't get outdated and trying to make the exception-handling work is a bit easier (no need to work in a separate branch and constantly rebase against master). What do you think?

I fully support this! And I'm glad that you changed your mind. My impressions was that you had the "all or nothing" approach, so that the ARM ABI bits could not land before we have the exceptions handling working too and before the ARM port is fully passing the Crystal test suite.

ssvb commented Oct 19, 2016

@asterite

Do you think we can still merge this anyway? In this way it doesn't get outdated and trying to make the exception-handling work is a bit easier (no need to work in a separate branch and constantly rebase against master). What do you think?

I fully support this! And I'm glad that you changed your mind. My impressions was that you had the "all or nothing" approach, so that the ARM ABI bits could not land before we have the exceptions handling working too and before the ARM port is fully passing the Crystal test suite.

@ssvb

This comment has been minimized.

Show comment
Hide comment
@ssvb

ssvb Oct 19, 2016

By the way, ARM development boards are very cheap nowadays. If the Raspberry Pi is too expensive, then I can recommend one of the Orange Pi boards. The shipping cost from China is very low. Also there are board variants with 2GB of RAM (this might be important for Crystal) and on-board WIFI / eMMC for a little bit higher price.

ssvb commented Oct 19, 2016

By the way, ARM development boards are very cheap nowadays. If the Raspberry Pi is too expensive, then I can recommend one of the Orange Pi boards. The shipping cost from China is very low. Also there are board variants with 2GB of RAM (this might be important for Crystal) and on-board WIFI / eMMC for a little bit higher price.

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 19, 2016

Member

@ssvb I needed the array type for my posix project, and the x86_64 ABI do references it, now.

I don't have a Raspberry yet, because I'm not sure which one to get. I wish there was a bunch of Zero with different processors (big or little endian, hard or soft float).

Member

ysbaddaden commented Oct 19, 2016

@ssvb I needed the array type for my posix project, and the x86_64 ABI do references it, now.

I don't have a Raspberry yet, because I'm not sure which one to get. I wish there was a bunch of Zero with different processors (big or little endian, hard or soft float).

@ssvb

This comment has been minimized.

Show comment
Hide comment
@ssvb

ssvb Oct 19, 2016

@ysbaddaden OK, thanks. If the array type is used now, then it makes sense.

As for big/little endian and hard/soft float, these are purely software things and you can try all of them on the same ARM processor. But big endian is a very unpopular choice and pretty much immature at the moment. As far as I know, NetBSD tried the big endian mode for fun (but I haven't tested it myself). Nowadays it may be a good idea to get a 64-bit ARMv8 board though.

ssvb commented Oct 19, 2016

@ysbaddaden OK, thanks. If the array type is used now, then it makes sense.

As for big/little endian and hard/soft float, these are purely software things and you can try all of them on the same ARM processor. But big endian is a very unpopular choice and pretty much immature at the moment. As far as I know, NetBSD tried the big endian mode for fun (but I haven't tested it myself). Nowadays it may be a good idea to get a 64-bit ARMv8 board though.

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 21, 2016

Member

Unwind is kinda different on ARM EHABI. The exception struct is different, the personality signature is different, the software is required to unwind the stack frame, ... Finding, then reading the GNU GCC __gxx_personality_v0 and Unwind implementation really helped to understand what the hell was happening.

Anyway, exceptions are now working on ARM EHABI. At least a begin/raise/rescue did work.

Member

ysbaddaden commented Oct 21, 2016

Unwind is kinda different on ARM EHABI. The exception struct is different, the personality signature is different, the software is required to unwind the stack frame, ... Finding, then reading the GNU GCC __gxx_personality_v0 and Unwind implementation really helped to understand what the hell was happening.

Anyway, exceptions are now working on ARM EHABI. At least a begin/raise/rescue did work.

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 22, 2016

Member

Unwind is almost working. Catch all rescues do catch the raised exceptions, but trying to catch a specific exception class doesn't. For example the following snippet doesn't work. It prints "Catch-All: Bar" instead of "Gotcha: Bar":

class Bar < Exception
end

def foo
  raise Bar.new("oh no")
end

begin
  foo
rescue ex : Bar
  puts "Gotcha: #{ex.class}"
rescue ex
  puts "Catch-All: #{ex.class}"
end

Note that in the catch-all handler, the object_id and crystal_type_id are correct and ex.is_a?(Bar) returns true 😭

Member

ysbaddaden commented Oct 22, 2016

Unwind is almost working. Catch all rescues do catch the raised exceptions, but trying to catch a specific exception class doesn't. For example the following snippet doesn't work. It prints "Catch-All: Bar" instead of "Gotcha: Bar":

class Bar < Exception
end

def foo
  raise Bar.new("oh no")
end

begin
  foo
rescue ex : Bar
  puts "Gotcha: #{ex.class}"
rescue ex
  puts "Catch-All: #{ex.class}"
end

Note that in the catch-all handler, the object_id and crystal_type_id are correct and ex.is_a?(Bar) returns true 😭

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 22, 2016

Member

Fixed! A stupid register error (I put the type in R2 when assembly expected R1).

Now, I have a segfault in spec/std/channel_spec.cr, more specifically in can be closed from different fiber. An exception is raised, but _Unwind_Backtrace segfaults when trying to collect the callstack. It's most likely caused by an issue in the fiber context switch.

Member

ysbaddaden commented Oct 22, 2016

Fixed! A stupid register error (I put the type in R2 when assembly expected R1).

Now, I have a segfault in spec/std/channel_spec.cr, more specifically in can be closed from different fiber. An exception is raised, but _Unwind_Backtrace segfaults when trying to collect the callstack. It's most likely caused by an issue in the fiber context switch.

@meew0

This comment has been minimized.

Show comment
Hide comment
@meew0

meew0 Oct 23, 2016

After the updates to this PR I've revisited the cross compilation I was doing before (see my earlier comments), here are my results:

The SecureRandom thing now works perfectly. Great, thanks!

The x86 compiler crash still exists, and the backtrace I get doesn't seem to have changed. The problem doesn't affect me that much and it may be unrelated to this PR, but I'd still be happy to provide any information necessary in fixing it.

I've also now tried to do the actual linking step on the Raspberry Pi itself, with the command output by the compiler on x86; and after installing and updating some missing libraries, building libcrystal.a and editing the command to point to it, I get the following output:

/usr/bin/ld: error: elgyem uses VFP register arguments, elgyem.o does not
/usr/bin/ld: failed to merge target specific data of file elgyem.o
collect2: error: ld returned 1 exit status

(In case there is confusion, elgyem is the name of the project.) From some Google searches, this should be fixable by adding -mfloat-abi=softfp (or soft or hard, I've tried all three) to the output, but nothing I've tried worked.

Apologies if this is something obvious I'm missing (I'm not very experienced in cross compilation) or if this problem is unrelated to the PR, but I feel like if this problem is fixable by adding a compiler flag, then that compiler flag should probably be added to the output when building on x86.

meew0 commented Oct 23, 2016

After the updates to this PR I've revisited the cross compilation I was doing before (see my earlier comments), here are my results:

The SecureRandom thing now works perfectly. Great, thanks!

The x86 compiler crash still exists, and the backtrace I get doesn't seem to have changed. The problem doesn't affect me that much and it may be unrelated to this PR, but I'd still be happy to provide any information necessary in fixing it.

I've also now tried to do the actual linking step on the Raspberry Pi itself, with the command output by the compiler on x86; and after installing and updating some missing libraries, building libcrystal.a and editing the command to point to it, I get the following output:

/usr/bin/ld: error: elgyem uses VFP register arguments, elgyem.o does not
/usr/bin/ld: failed to merge target specific data of file elgyem.o
collect2: error: ld returned 1 exit status

(In case there is confusion, elgyem is the name of the project.) From some Google searches, this should be fixable by adding -mfloat-abi=softfp (or soft or hard, I've tried all three) to the output, but nothing I've tried worked.

Apologies if this is something obvious I'm missing (I'm not very experienced in cross compilation) or if this problem is unrelated to the PR, but I feel like if this problem is fixable by adding a compiler flag, then that compiler flag should probably be added to the output when building on x86.

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 23, 2016

Member

Thanks for testing. Please do!

I didn't tested on a real Pi. I only did on QEMU with the versatilepb machine, arm1176 CPU, 256M of RAM and the latest Raspbian image, which targets ARMv6 with hard float. That is a Pi A or Pi Zero (I'm waiting for thepihut to stock some zeros to test on real hardware).

That being said, I never got a soft/hard float issue. I cross compile with --cross-compile --target=arm-unknown-linux-gnueabihf to get an object file, then link it in Raspbian/QEMU. Maybe toying with the target triple, so LLVM knows what features to use could help?

Member

ysbaddaden commented Oct 23, 2016

Thanks for testing. Please do!

I didn't tested on a real Pi. I only did on QEMU with the versatilepb machine, arm1176 CPU, 256M of RAM and the latest Raspbian image, which targets ARMv6 with hard float. That is a Pi A or Pi Zero (I'm waiting for thepihut to stock some zeros to test on real hardware).

That being said, I never got a soft/hard float issue. I cross compile with --cross-compile --target=arm-unknown-linux-gnueabihf to get an object file, then link it in Raspbian/QEMU. Maybe toying with the target triple, so LLVM knows what features to use could help?

@meew0

This comment has been minimized.

Show comment
Hide comment
@meew0

meew0 Oct 23, 2016

Ah, it seems like using arm-unknown-linux-gnueabihf works; I've only been using arm-linux-gnueabihf previously. Now it compiles and links successfully, thanks!

However I'm now getting a segfault at runtime:

Invalid memory access (signal 11) at address 0x1c
[1519956] ???
[475236] __crystal_sigfault_handler +64

This trace alone probably isn't of much use to you; tell me if you need any more information. I've been able to trace it down to a minimal example that simply makes an HTTP request: https://gist.github.com/meew0/c0683f76f603f8c8da135f9264245ffc

(Note that this specific example produces a slightly different segfault, however I don't think that matters much.)

meew0 commented Oct 23, 2016

Ah, it seems like using arm-unknown-linux-gnueabihf works; I've only been using arm-linux-gnueabihf previously. Now it compiles and links successfully, thanks!

However I'm now getting a segfault at runtime:

Invalid memory access (signal 11) at address 0x1c
[1519956] ???
[475236] __crystal_sigfault_handler +64

This trace alone probably isn't of much use to you; tell me if you need any more information. I've been able to trace it down to a minimal example that simply makes an HTTP request: https://gist.github.com/meew0/c0683f76f603f8c8da135f9264245ffc

(Note that this specific example produces a slightly different segfault, however I don't think that matters much.)

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 23, 2016

Member

@meew0 you may comment out LibExt.setup_sigfault_handler in src/signal.cr and compile with --debug then use gdb to get a backtrace. Yet, I have a feeling the fiber context switch ASM is the culprit.

Member

ysbaddaden commented Oct 23, 2016

@meew0 you may comment out LibExt.setup_sigfault_handler in src/signal.cr and compile with --debug then use gdb to get a backtrace. Yet, I have a feeling the fiber context switch ASM is the culprit.

@meew0

This comment has been minimized.

Show comment
Hide comment
@meew0

meew0 Oct 23, 2016

This is what I got when running the minimal example I posted above, with my very limited gdb experience:

(gdb) run
Starting program: /home/pi/minimal2 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) backtrace
#0  0x00000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

meew0 commented Oct 23, 2016

This is what I got when running the minimal example I posted above, with my very limited gdb experience:

(gdb) run
Starting program: /home/pi/minimal2 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) backtrace
#0  0x00000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 23, 2016

Member

I found in the D runtime the answer to my unwind crash (I believe), and also notes on which registers should actually be saved (r4-r11, lr, d8-d15):
https://github.com/dlang/druntime/blob/master/src/core/threadasm.S#L438

I'm working on that, and need to get my head around registers, stacks, etc. Done!

Member

ysbaddaden commented Oct 23, 2016

I found in the D runtime the answer to my unwind crash (I believe), and also notes on which registers should actually be saved (r4-r11, lr, d8-d15):
https://github.com/dlang/druntime/blob/master/src/core/threadasm.S#L438

I'm working on that, and need to get my head around registers, stacks, etc. Done!

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 24, 2016

Member

@meew0 I just fixed the fiber context switch segfault, and it fixed your reduced test case!

I can now run the std spec suite without any crash. It merely hangs a bit when computing the BCrypt digests, which is unavoidable.

./std_spec
Finished in 1:07 minutes
3451 examples, 187 failures, 69 errors, 1 pending

Member

ysbaddaden commented Oct 24, 2016

@meew0 I just fixed the fiber context switch segfault, and it fixed your reduced test case!

I can now run the std spec suite without any crash. It merely hangs a bit when computing the BCrypt digests, which is unavoidable.

./std_spec
Finished in 1:07 minutes
3451 examples, 187 failures, 69 errors, 1 pending

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 24, 2016

Member

After fixing the path at which std specs expect some files to be, I now have:

4466 examples, 194 failures, 5 errors, 3 pending

Of which there are:

  • 1 unwind error (failed to raise an XML::Error exception) —no clue, the callstack is popoulated corectly, but when raising, the personality routine is never called;
  • 8 socket errors because my QEMU machine lacks IPv6 support —we'll may want specs to check for IPv6 support;
  • (almost) everything else are Float errors: the generated code is failing on every FPU calculations —no clue, yet.
Member

ysbaddaden commented Oct 24, 2016

After fixing the path at which std specs expect some files to be, I now have:

4466 examples, 194 failures, 5 errors, 3 pending

Of which there are:

  • 1 unwind error (failed to raise an XML::Error exception) —no clue, the callstack is popoulated corectly, but when raising, the personality routine is never called;
  • 8 socket errors because my QEMU machine lacks IPv6 support —we'll may want specs to check for IPv6 support;
  • (almost) everything else are Float errors: the generated code is failing on every FPU calculations —no clue, yet.
@meew0

This comment has been minimized.

Show comment
Hide comment
@meew0

meew0 Oct 24, 2016

Hm, when trying to cross-compile, I now get the following error (on x86):

Codegen (crystal):                 00:00:00.0136101 (  35.99MB)
<inline asm>:4:9: error: instruction requires: VFP2
        vstmdb sp!, {d8-d15}
        ^
<inline asm>:7:9: error: instruction requires: VFP2
        vldmia sp!, {d8-d15}
        ^
LLVM ERROR: Error parsing inline asm

It happens with anything I'm trying to cross-compile, even an empty file. Is it something with the target triple again? It happens with both arm-unknown-linux-gnueabihf and arm-linux-gnueabihf.

meew0 commented Oct 24, 2016

Hm, when trying to cross-compile, I now get the following error (on x86):

Codegen (crystal):                 00:00:00.0136101 (  35.99MB)
<inline asm>:4:9: error: instruction requires: VFP2
        vstmdb sp!, {d8-d15}
        ^
<inline asm>:7:9: error: instruction requires: VFP2
        vldmia sp!, {d8-d15}
        ^
LLVM ERROR: Error parsing inline asm

It happens with anything I'm trying to cross-compile, even an empty file. Is it something with the target triple again? It happens with both arm-unknown-linux-gnueabihf and arm-linux-gnueabihf.

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 24, 2016

Member

No, it's the fiber context switch. I now push/pop the FPU registers too. LLVM complained for me, too, but I added .fpu vfp to the ASM (see) and it stopped complaining. I compiled Crystal against LLVM 3.9 (with ARM codegen fixes) by-the-way. Or maybe we must specify another version, like .fpu vfpv2?

Member

ysbaddaden commented Oct 24, 2016

No, it's the fiber context switch. I now push/pop the FPU registers too. LLVM complained for me, too, but I added .fpu vfp to the ASM (see) and it stopped complaining. I compiled Crystal against LLVM 3.9 (with ARM codegen fixes) by-the-way. Or maybe we must specify another version, like .fpu vfpv2?

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 24, 2016

Member

@asterite the FPU is working. LLVM intrinsics are fine with both Float32 and Float64, Crystal is fine with Float32, but fails when dealing with Float64. Note that ARM VFP (vector-floating-point) uses double word registers.

Failing example:

a = 123.456_f32
p a.ceil           #=> 124 (Float32)
p LLVM.ceil_f32(a) #=> 124 (Float32)

b = 123.456_f64
p b.ceil           #=> 1.273198284006911e-313 (Float64)
p LLVM.ceil_f64(b) #=> 124.0 (Float64)

If I prepend pp self to Float64#ceil then b.ceil becomes correct. Could LLVM fail to understand that self is a double primitive?

Member

ysbaddaden commented Oct 24, 2016

@asterite the FPU is working. LLVM intrinsics are fine with both Float32 and Float64, Crystal is fine with Float32, but fails when dealing with Float64. Note that ARM VFP (vector-floating-point) uses double word registers.

Failing example:

a = 123.456_f32
p a.ceil           #=> 124 (Float32)
p LLVM.ceil_f32(a) #=> 124 (Float32)

b = 123.456_f64
p b.ceil           #=> 1.273198284006911e-313 (Float64)
p LLVM.ceil_f64(b) #=> 124.0 (Float64)

If I prepend pp self to Float64#ceil then b.ceil becomes correct. Could LLVM fail to understand that self is a double primitive?

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Oct 24, 2016

Member

If I compile with --release, then the bug disappears for something as simple as ceil, because the calls are inlined (no more Float64#ceil symbol, nor bl <ceil> in the disassembly).

The std spec is down to (when compiled with --release):

4466 examples, 96 failures, 4 errors, 3 pending

But the issue is still there for methods of Math that aren't inlined. For example expm1, log2 or acosh.

Member

ysbaddaden commented Oct 24, 2016

If I compile with --release, then the bug disappears for something as simple as ceil, because the calls are inlined (no more Float64#ceil symbol, nor bl <ceil> in the disassembly).

The std spec is down to (when compiled with --release):

4466 examples, 96 failures, 4 errors, 3 pending

But the issue is still there for methods of Math that aren't inlined. For example expm1, log2 or acosh.

@meew0

This comment has been minimized.

Show comment
Hide comment
@meew0

meew0 Oct 24, 2016

After some messing around with LLVM versions (I was on 3.5 previously), everything works perfectly now! The x86 segfault when compiling in release mode seems to be gone as well. Thanks a lot!

meew0 commented Oct 24, 2016

After some messing around with LLVM versions (I was on 3.5 previously), everything works perfectly now! The x86 segfault when compiling in release mode seems to be gone as well. Thanks a lot!

@asterite

This comment has been minimized.

Show comment
Hide comment
@asterite

asterite Oct 25, 2016

Contributor

@ysbaddaden Does it work if you compile with --mcpu=cortex-a9?

That question has a reason, I'll leave the original text I was going to write before finding the "answer":


Findings so far...

I'm using this program, which can be compiled with --prelude=empty:

{% if flag?(:linux) || flag?(:freebsd) || flag?(:openbsd) %}
  @[Link("m")]
{% end %}
lib LibC
  fun printf(x0 : UInt8*, ...) : Int32
end

lib LibM
  fun ceil_f64 = "llvm.ceil.f64"(value : Float64) : Float64
end

class String
  def to_unsafe
    pointerof(@c)
  end
end

def foo(b)
  LibM.ceil_f64(b)
end

b = 123.456_f64
b = foo(b)
LibC.printf "%g\n", b

I made a C program and compiled it with clang to see what it generated (this is with clang 3.0 on a real raspberry pi 2):

#include <stdio.h>
#include <math.h>

double doit(double x) {
  return ceil(x);
}

int main() {
  double x = 123.456;
  x = doit(x);
  printf("%g\n", x);
  return 0;
}

The LLVM IR for that, on the raspberry, is:

; ModuleID = 'c.c'
target datalayout = "e-p:32:32:32-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-f32:32:32-f64:32:64-v64:32:64-v128:32:128-a0:0:32-n32-S32"
target triple = "armv4t-unknown-linux-gnueabihf"

@.str = private unnamed_addr constant [4 x i8] c"%g\0A\00", align 1

define double @doit(double %x) nounwind {
  %1 = alloca double, align 4
  store double %x, double* %1, align 8
  %2 = load double* %1, align 8
  %3 = call double @ceil(double %2) nounwind readnone
  ret double %3
}

declare double @ceil(double) nounwind readnone

define i32 @main() nounwind {
  %1 = alloca i32, align 4
  %x = alloca double, align 8
  store i32 0, i32* %1
  store double 1.234560e+02, double* %x, align 8
  %2 = load double* %x, align 8
  %3 = call double @doit(double %2)
  store double %3, double* %x, align 8
  %4 = load double* %x, align 8
  %5 = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([4 x i8]* @.str, i32 0, i32 0), double %4)
  ret i32 0
}

declare i32 @printf(i8*, ...)

One difference: the target triple is "armv4t-unknown-linux-gnueabihf".

However, then I try to do this on my machine:

$ bin/crystal build --cross-compile -D arm --target arm-unknown-linux-gnueabihf --prelude=empty foo.cr --link-flags="-mcpu=cortex-a9"
cc foo.o -o foo -mcpu=cortex-a9 -rdynamic  -lm -L/usr/lib -L/usr/local/lib

(the -mcpu=cortex-a9 seems to be needed, explanation below)

Now I copy that foo.o and link it and run it:

$ cc foo.o -o foo -mcpu=cortex-a9 -rdynamic  -lm -L/usr/lib -L/usr/local/lib && ./foo
0

Nope :-(

However, if I generate an LLVM IR file for foo.cr:

$ bin/crystal build -D arm --target armv4t-unknown-linux-gnueabihf --prelude=empty foo.cr --emit llvm-ir

(it gives a warning, but no problem)

Now the LLVM IR is this:

; ModuleID = 'main_module'
source_filename = "main_module"
target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64"
target triple = "armv4t-unknown-linux-gnueabihf"

%String = type { i32, i32, i32, i8 }

@ARGC_UNSAFE = internal global i32 0
@ARGV_UNSAFE = internal global i8** null
@"'%g\0A'" = private constant { i32, i32, i32, [4 x i8] } { i32 1, i32 3, i32 3, [4 x i8] c"%g\0A\00" }

define internal i32 @__crystal_main(i32 %argc, i8** %argv) {
alloca:
  %b = alloca double
  br label %entry

entry:                                            ; preds = %alloca
  store i32 %argc, i32* @ARGC_UNSAFE
  store i8** %argv, i8*** @ARGV_UNSAFE
  store double 1.234560e+02, double* %b
  %0 = load double, double* %b
  %1 = call double @"*foo<Float64>:Float64"(double %0)
  store double %1, double* %b
  %2 = call i8* @"*String#to_unsafe:Pointer(UInt8)"(%String* bitcast ({ i32, i32, i32, [4 x i8] }* @"'%g\0A'" to %String*))
  %3 = load double, double* %b
  %4 = call i32 (i8*, ...) @printf(i8* %2, double %3)
  ret i32 %4
}

declare i32 @printf(i8*, ...)

; Function Attrs: uwtable
define i32 @main(i32 %argc, i8** %argv) #0 {
entry:
  %0 = call i32 @__crystal_main(i32 %argc, i8** %argv)
  ret i32 0
}

; Function Attrs: uwtable
define internal double @"*foo<Float64>:Float64"(double %b) #0 {
entry:
  %0 = call double @llvm.ceil.f64(double %b)
  ret double %0
}

; Function Attrs: nounwind readnone
declare double @llvm.ceil.f64(double) #1

; Function Attrs: uwtable
define internal i8* @"*String#to_unsafe:Pointer(UInt8)"(%String* %self) #0 {
entry:
  %0 = getelementptr inbounds %String, %String* %self, i32 0, i32 3
  ret i8* %0
}

; Function Attrs: nounwind
declare void @llvm.stackprotector(i8*, i8**) #2

attributes #0 = { uwtable }
attributes #1 = { nounwind readnone }
attributes #2 = { nounwind }

I link it:

$ llc foo.ll -filetype=obj -o foo.o -float-abi=hard -mcpu=cortex-a9

Then copy foo.o to the raspberry and link it and run it again:

$ cc foo.o -o foo -rdynamic  -lm -L/usr/lib -L/usr/local/lib && ./foo
124

It... works??!

I don't know why is there a difference, but maybe it helps to figure out the problem.

About -mcpu=cortex-a9: if I don't put it in the llc command then it doesn't work, it prints 0 again.

Another maybe notable thing is that clang for ARM moves the double parameter to an alloca that has align 4, and stores/loads from it using align 8. I don't know if it is relevant, I spent some time thinking that was it, but apparently not.

Yet another difference is that clang uses ceil while we use llvm.ceil.f64, but again it doesn't seem to make a difference.

FINALLY: I then remember that you can specify the cpu with --mcpu, so the --link-flags wasn't needed, and in fact wasn't working (the cpu is used in the object generation in our code). With that:

$ bin/crystal build --cross-compile --target armv4t-unknown-linux-gnueabihf --prelude=empty foo.cr --mcpu=cortex-a9
cc foo.o -o foo  -rdynamic  -lm -L/usr/lib -L/usr/local/lib

Copy over the raspberry pi:

$ cc foo.o -o foo -mcpu=cortex-a9 -rdynamic  -lm -L/usr/lib -L/usr/local/lib && ./foo
124

YAY!! :-)

So using this mcpu seems to fix it, but no idea why... (I read it here)

Contributor

asterite commented Oct 25, 2016

@ysbaddaden Does it work if you compile with --mcpu=cortex-a9?

That question has a reason, I'll leave the original text I was going to write before finding the "answer":


Findings so far...

I'm using this program, which can be compiled with --prelude=empty:

{% if flag?(:linux) || flag?(:freebsd) || flag?(:openbsd) %}
  @[Link("m")]
{% end %}
lib LibC
  fun printf(x0 : UInt8*, ...) : Int32
end

lib LibM
  fun ceil_f64 = "llvm.ceil.f64"(value : Float64) : Float64
end

class String
  def to_unsafe
    pointerof(@c)
  end
end

def foo(b)
  LibM.ceil_f64(b)
end

b = 123.456_f64
b = foo(b)
LibC.printf "%g\n", b

I made a C program and compiled it with clang to see what it generated (this is with clang 3.0 on a real raspberry pi 2):

#include <stdio.h>
#include <math.h>

double doit(double x) {
  return ceil(x);
}

int main() {
  double x = 123.456;
  x = doit(x);
  printf("%g\n", x);
  return 0;
}

The LLVM IR for that, on the raspberry, is:

; ModuleID = 'c.c'
target datalayout = "e-p:32:32:32-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-f32:32:32-f64:32:64-v64:32:64-v128:32:128-a0:0:32-n32-S32"
target triple = "armv4t-unknown-linux-gnueabihf"

@.str = private unnamed_addr constant [4 x i8] c"%g\0A\00", align 1

define double @doit(double %x) nounwind {
  %1 = alloca double, align 4
  store double %x, double* %1, align 8
  %2 = load double* %1, align 8
  %3 = call double @ceil(double %2) nounwind readnone
  ret double %3
}

declare double @ceil(double) nounwind readnone

define i32 @main() nounwind {
  %1 = alloca i32, align 4
  %x = alloca double, align 8
  store i32 0, i32* %1
  store double 1.234560e+02, double* %x, align 8
  %2 = load double* %x, align 8
  %3 = call double @doit(double %2)
  store double %3, double* %x, align 8
  %4 = load double* %x, align 8
  %5 = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([4 x i8]* @.str, i32 0, i32 0), double %4)
  ret i32 0
}

declare i32 @printf(i8*, ...)

One difference: the target triple is "armv4t-unknown-linux-gnueabihf".

However, then I try to do this on my machine:

$ bin/crystal build --cross-compile -D arm --target arm-unknown-linux-gnueabihf --prelude=empty foo.cr --link-flags="-mcpu=cortex-a9"
cc foo.o -o foo -mcpu=cortex-a9 -rdynamic  -lm -L/usr/lib -L/usr/local/lib

(the -mcpu=cortex-a9 seems to be needed, explanation below)

Now I copy that foo.o and link it and run it:

$ cc foo.o -o foo -mcpu=cortex-a9 -rdynamic  -lm -L/usr/lib -L/usr/local/lib && ./foo
0

Nope :-(

However, if I generate an LLVM IR file for foo.cr:

$ bin/crystal build -D arm --target armv4t-unknown-linux-gnueabihf --prelude=empty foo.cr --emit llvm-ir

(it gives a warning, but no problem)

Now the LLVM IR is this:

; ModuleID = 'main_module'
source_filename = "main_module"
target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64"
target triple = "armv4t-unknown-linux-gnueabihf"

%String = type { i32, i32, i32, i8 }

@ARGC_UNSAFE = internal global i32 0
@ARGV_UNSAFE = internal global i8** null
@"'%g\0A'" = private constant { i32, i32, i32, [4 x i8] } { i32 1, i32 3, i32 3, [4 x i8] c"%g\0A\00" }

define internal i32 @__crystal_main(i32 %argc, i8** %argv) {
alloca:
  %b = alloca double
  br label %entry

entry:                                            ; preds = %alloca
  store i32 %argc, i32* @ARGC_UNSAFE
  store i8** %argv, i8*** @ARGV_UNSAFE
  store double 1.234560e+02, double* %b
  %0 = load double, double* %b
  %1 = call double @"*foo<Float64>:Float64"(double %0)
  store double %1, double* %b
  %2 = call i8* @"*String#to_unsafe:Pointer(UInt8)"(%String* bitcast ({ i32, i32, i32, [4 x i8] }* @"'%g\0A'" to %String*))
  %3 = load double, double* %b
  %4 = call i32 (i8*, ...) @printf(i8* %2, double %3)
  ret i32 %4
}

declare i32 @printf(i8*, ...)

; Function Attrs: uwtable
define i32 @main(i32 %argc, i8** %argv) #0 {
entry:
  %0 = call i32 @__crystal_main(i32 %argc, i8** %argv)
  ret i32 0
}

; Function Attrs: uwtable
define internal double @"*foo<Float64>:Float64"(double %b) #0 {
entry:
  %0 = call double @llvm.ceil.f64(double %b)
  ret double %0
}

; Function Attrs: nounwind readnone
declare double @llvm.ceil.f64(double) #1

; Function Attrs: uwtable
define internal i8* @"*String#to_unsafe:Pointer(UInt8)"(%String* %self) #0 {
entry:
  %0 = getelementptr inbounds %String, %String* %self, i32 0, i32 3
  ret i8* %0
}

; Function Attrs: nounwind
declare void @llvm.stackprotector(i8*, i8**) #2

attributes #0 = { uwtable }
attributes #1 = { nounwind readnone }
attributes #2 = { nounwind }

I link it:

$ llc foo.ll -filetype=obj -o foo.o -float-abi=hard -mcpu=cortex-a9

Then copy foo.o to the raspberry and link it and run it again:

$ cc foo.o -o foo -rdynamic  -lm -L/usr/lib -L/usr/local/lib && ./foo
124

It... works??!

I don't know why is there a difference, but maybe it helps to figure out the problem.

About -mcpu=cortex-a9: if I don't put it in the llc command then it doesn't work, it prints 0 again.

Another maybe notable thing is that clang for ARM moves the double parameter to an alloca that has align 4, and stores/loads from it using align 8. I don't know if it is relevant, I spent some time thinking that was it, but apparently not.

Yet another difference is that clang uses ceil while we use llvm.ceil.f64, but again it doesn't seem to make a difference.

FINALLY: I then remember that you can specify the cpu with --mcpu, so the --link-flags wasn't needed, and in fact wasn't working (the cpu is used in the object generation in our code). With that:

$ bin/crystal build --cross-compile --target armv4t-unknown-linux-gnueabihf --prelude=empty foo.cr --mcpu=cortex-a9
cc foo.o -o foo  -rdynamic  -lm -L/usr/lib -L/usr/local/lib

Copy over the raspberry pi:

$ cc foo.o -o foo -mcpu=cortex-a9 -rdynamic  -lm -L/usr/lib -L/usr/local/lib && ./foo
124

YAY!! :-)

So using this mcpu seems to fix it, but no idea why... (I read it here)

@ysbaddaden ysbaddaden referenced this pull request Oct 31, 2016

Merged

AArch64 support (ARMv8-A) #3491

11 of 11 tasks complete
@zeiv

This comment has been minimized.

Show comment
Hide comment
@zeiv

zeiv Nov 1, 2016

Can you explain how to compile crystal for arm? I'm getting (on mac) Error: llc: No available targets are compatible with this triple. for arm-unknown-linux-gnueabihf.

I passed in the following:
crystal build --cross-compile --target=arm-unknown-linux-gnueabihf --prelude=empty src/compiler/crystal.cr

zeiv commented Nov 1, 2016

Can you explain how to compile crystal for arm? I'm getting (on mac) Error: llc: No available targets are compatible with this triple. for arm-unknown-linux-gnueabihf.

I passed in the following:
crystal build --cross-compile --target=arm-unknown-linux-gnueabihf --prelude=empty src/compiler/crystal.cr

@RX14

This comment has been minimized.

Show comment
Hide comment
@RX14

RX14 Nov 1, 2016

Member

@zeiv are you sure you're running crystal built from master? This PR isn't in the latest release.

Member

RX14 commented Nov 1, 2016

@zeiv are you sure you're running crystal built from master? This PR isn't in the latest release.

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Nov 2, 2016

Member

@zeiv you must first compile the latest master of Crystal, then you'll be able to target ARM (or wait for the 0.20 to be released). Also, do not specify an empty prelude, std and core lib are fine, and they're required to compile Crystal.

You may either cross compile Crystal itself (again, latest master). Be sure to have the same LLVM version on both the host and the guest. You'll may also want to install/compile a recent LLVM version (eg: 3.8 or 3.9). Be sure to not pass an empty prelude (compilation with fail).

bin/crystal build src/compiler/crystal.cr \
  --cross-compile --target arm-unknown-linux-gnueabihf \
  --release -Dwithout_openssl -Dwithout_zlib

Or you may cross-compile your applications for ARM:

bin/crystal build myapp.cr \
  --cross-compile --target arm-unknown-linux-gnueabihf

Once an object file is created, copy it to the the ARM board (or VM), but before linking it by copy/pasting the printed cc command, you'll need a bunch of libraries to be installed on the ARM board. Please refer to Required Libraries for an exhausted list. You'll have to compile Boehm GC 7.6; clone crystal and run make deps to compile the src/llvm/ext/llvm_ext.o and src/ext/libcrystal.a objects.

Eventually tweak the copied cc command to refer correctly to llvm_ext.o and libcrystal.a, and you should have a working Crystal compiler or application running on ARM.

Member

ysbaddaden commented Nov 2, 2016

@zeiv you must first compile the latest master of Crystal, then you'll be able to target ARM (or wait for the 0.20 to be released). Also, do not specify an empty prelude, std and core lib are fine, and they're required to compile Crystal.

You may either cross compile Crystal itself (again, latest master). Be sure to have the same LLVM version on both the host and the guest. You'll may also want to install/compile a recent LLVM version (eg: 3.8 or 3.9). Be sure to not pass an empty prelude (compilation with fail).

bin/crystal build src/compiler/crystal.cr \
  --cross-compile --target arm-unknown-linux-gnueabihf \
  --release -Dwithout_openssl -Dwithout_zlib

Or you may cross-compile your applications for ARM:

bin/crystal build myapp.cr \
  --cross-compile --target arm-unknown-linux-gnueabihf

Once an object file is created, copy it to the the ARM board (or VM), but before linking it by copy/pasting the printed cc command, you'll need a bunch of libraries to be installed on the ARM board. Please refer to Required Libraries for an exhausted list. You'll have to compile Boehm GC 7.6; clone crystal and run make deps to compile the src/llvm/ext/llvm_ext.o and src/ext/libcrystal.a objects.

Eventually tweak the copied cc command to refer correctly to llvm_ext.o and libcrystal.a, and you should have a working Crystal compiler or application running on ARM.

@zeiv

This comment has been minimized.

Show comment
Hide comment
@zeiv

zeiv Nov 2, 2016

@ysbaddaden Thanks for the reply. That's what I was missing, I was using bin/crystal from master without actually compiling the entire source. I'm trying to compile master on my mac and am now running into the following error:

$ make LLVM_CONFIG=/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 
Using /usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 [version=3.6.2]
c++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc `/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 --cxxflags`
cc -fPIC    -c -o src/ext/sigfault.o src/ext/sigfault.c
ar -rcs src/ext/libcrystal.a src/ext/sigfault.o
CRYSTAL_CONFIG_PATH=`pwd`/src ./bin/crystal build  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Undefined symbols for architecture x86_64:
  "_LLVMInitializeARMAsmParser", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMAsmPrinter", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTarget", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTargetInfo", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTargetMC", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: execution of command failed with code: 1: `cc -o "/Users/xavier/OneDrive/Documents/Dev/crystal/crystal/.build/crystal" "${@}"  -rdynamic  /Users/xavier/OneDrive/Documents/Dev/crystal/crystal/src/llvm/ext/llvm_ext.o `/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lgc -lpthread /Users/xavier/OneDrive/Documents/Dev/crystal/crystal/src/ext/libcrystal.a -levent -liconv -ldl -L/usr/lib -L/usr/local/lib`
make: *** [.build/crystal] Error 

zeiv commented Nov 2, 2016

@ysbaddaden Thanks for the reply. That's what I was missing, I was using bin/crystal from master without actually compiling the entire source. I'm trying to compile master on my mac and am now running into the following error:

$ make LLVM_CONFIG=/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 
Using /usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 [version=3.6.2]
c++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc `/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 --cxxflags`
cc -fPIC    -c -o src/ext/sigfault.o src/ext/sigfault.c
ar -rcs src/ext/libcrystal.a src/ext/sigfault.o
CRYSTAL_CONFIG_PATH=`pwd`/src ./bin/crystal build  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Undefined symbols for architecture x86_64:
  "_LLVMInitializeARMAsmParser", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMAsmPrinter", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTarget", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTargetInfo", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
  "_LLVMInitializeARMTargetMC", referenced from:
      _*LLVM::init_arm:Nil in L-L-V-M-.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: execution of command failed with code: 1: `cc -o "/Users/xavier/OneDrive/Documents/Dev/crystal/crystal/.build/crystal" "${@}"  -rdynamic  /Users/xavier/OneDrive/Documents/Dev/crystal/crystal/src/llvm/ext/llvm_ext.o `/usr/local/Cellar/llvm36/3.6.2/bin/llvm-config-3.6 --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lgc -lpthread /Users/xavier/OneDrive/Documents/Dev/crystal/crystal/src/ext/libcrystal.a -levent -liconv -ldl -L/usr/lib -L/usr/local/lib`
make: *** [.build/crystal] Error 
@zeiv

This comment has been minimized.

Show comment
Hide comment
@zeiv

zeiv Nov 2, 2016

UPDATE
I was able to successfully cross-compile the compiler WITHOUT the --release option using Ubuntu.

I was compiling on a digital ocean droplet. When I first tried to compile with --release, this happened...

$ bin/crystal build src/compiler/crystal.cr --cross-compile --target arm-unknown-linux-gnueabihf --release -Dwithout_openssl -Dwithout_zlib
Using compiled compiler at .build/crystal

Invalid memory access (signal 11) at address 0x18
[9183605] *CallStack::print_backtrace:Int32 +117
[9062216] __crystal_sigfault_handler +56
[35486375] sigfault_handler +40
[140131403375584] ???
[26166049] _ZN4llvm12SelectionDAG7getNodeEjNS_5SDLocENS_3EVTENS_7SDValueE +65
[26947182] ???
[26949563] ???
[26603809] ???
[26604895] _ZN4llvm12SelectionDAG13LegalizeTypesEv +943
[26456599] _ZN4llvm16SelectionDAGISel17CodeGenAndEmitDAGEv +215
[26470802] _ZN4llvm16SelectionDAGISel20SelectAllBasicBlocksERKNS_8FunctionE +1058
[26476045] _ZN4llvm16SelectionDAGISel20runOnMachineFunctionERNS_15MachineFunctionE +669
[23793557] ???
[34592087] _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE +487
[34592747] _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE +43
[34591156] _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE +772
[33208958] ???
[33210344] LLVMTargetMachineEmitToFile +456
[21001963] *LLVM::TargetMachine#emit_to_file<LLVM::Module, String, LLVM::CodeGenFileType>:Bool +91
[21001851] *LLVM::TargetMachine#emit_obj_to_file<LLVM::Module, String>:Bool +11
[17811350] *Crystal::Compiler#cross_compile<Crystal::Program, Array(Crystal::Compiler::CompilationUnit), String, String>:Nil +326
[17810546] *Crystal::Compiler#codegen<Crystal::Program, Crystal::ASTNode+, Array(Crystal::Compiler::Source), String>:(Array(String) | Nil) +1746
[17800966] *Crystal::Compiler#compile<Array(Crystal::Compiler::Source), String>:Crystal::Compiler::Result +150
[17825657] *Crystal::Command::CompilerConfig#compile<String>:Crystal::Compiler::Result +57
[17825570] *Crystal::Command::CompilerConfig#compile:Crystal::Compiler::Result +34
[10002466] *Crystal::Command#build:Crystal::Compiler::Result +290
[9999371] *Crystal::Command#run:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +491
[9998603] *Crystal::Command::run<Array(String)>:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +27
[9998537] *Crystal::Command::run:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +25
[8987984] ???
[9061913] main +41
[140131383912496] __libc_start_main +240
[8984489] _start +41
[0] ???

I thought it might be a memory leak or something, so I resized my droplet to their super-duper 64GB / 16 core option and tried again, but it crashed again. (I was using screen and it terminated when it died, so I don't have a trace unfortunately).

zeiv commented Nov 2, 2016

UPDATE
I was able to successfully cross-compile the compiler WITHOUT the --release option using Ubuntu.

I was compiling on a digital ocean droplet. When I first tried to compile with --release, this happened...

$ bin/crystal build src/compiler/crystal.cr --cross-compile --target arm-unknown-linux-gnueabihf --release -Dwithout_openssl -Dwithout_zlib
Using compiled compiler at .build/crystal

Invalid memory access (signal 11) at address 0x18
[9183605] *CallStack::print_backtrace:Int32 +117
[9062216] __crystal_sigfault_handler +56
[35486375] sigfault_handler +40
[140131403375584] ???
[26166049] _ZN4llvm12SelectionDAG7getNodeEjNS_5SDLocENS_3EVTENS_7SDValueE +65
[26947182] ???
[26949563] ???
[26603809] ???
[26604895] _ZN4llvm12SelectionDAG13LegalizeTypesEv +943
[26456599] _ZN4llvm16SelectionDAGISel17CodeGenAndEmitDAGEv +215
[26470802] _ZN4llvm16SelectionDAGISel20SelectAllBasicBlocksERKNS_8FunctionE +1058
[26476045] _ZN4llvm16SelectionDAGISel20runOnMachineFunctionERNS_15MachineFunctionE +669
[23793557] ???
[34592087] _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE +487
[34592747] _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE +43
[34591156] _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE +772
[33208958] ???
[33210344] LLVMTargetMachineEmitToFile +456
[21001963] *LLVM::TargetMachine#emit_to_file<LLVM::Module, String, LLVM::CodeGenFileType>:Bool +91
[21001851] *LLVM::TargetMachine#emit_obj_to_file<LLVM::Module, String>:Bool +11
[17811350] *Crystal::Compiler#cross_compile<Crystal::Program, Array(Crystal::Compiler::CompilationUnit), String, String>:Nil +326
[17810546] *Crystal::Compiler#codegen<Crystal::Program, Crystal::ASTNode+, Array(Crystal::Compiler::Source), String>:(Array(String) | Nil) +1746
[17800966] *Crystal::Compiler#compile<Array(Crystal::Compiler::Source), String>:Crystal::Compiler::Result +150
[17825657] *Crystal::Command::CompilerConfig#compile<String>:Crystal::Compiler::Result +57
[17825570] *Crystal::Command::CompilerConfig#compile:Crystal::Compiler::Result +34
[10002466] *Crystal::Command#build:Crystal::Compiler::Result +290
[9999371] *Crystal::Command#run:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +491
[9998603] *Crystal::Command::run<Array(String)>:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +27
[9998537] *Crystal::Command::run:(Array(Crystal::ImplementationTrace) | Array(Crystal::Init::View+:Class) | Array(String) | Bool | Crystal::Compiler::Result | Hash(String, String) | IO::FileDescriptor | Nil) +25
[8987984] ???
[9061913] main +41
[140131383912496] __libc_start_main +240
[8984489] _start +41
[0] ???

I thought it might be a memory leak or something, so I resized my droplet to their super-duper 64GB / 16 core option and tried again, but it crashed again. (I was using screen and it terminated when it died, so I don't have a trace unfortunately).

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Nov 2, 2016

Member

@zeiv homebrew is missing some targets by default. AArch64 isn't enabled for 3.9 and it seems ARM wasn't in 3.6 :-(

I'll try to reproduce the crash with --release. It's a crash in LLVM.

Member

ysbaddaden commented Nov 2, 2016

@zeiv homebrew is missing some targets by default. AArch64 isn't enabled for 3.9 and it seems ARM wasn't in 3.6 :-(

I'll try to reproduce the crash with --release. It's a crash in LLVM.

@meew0

This comment has been minimized.

Show comment
Hide comment
@meew0

meew0 Nov 2, 2016

That segfault looks like the one I got; it was fixed by upgrading LLVM to 3.9.

meew0 commented Nov 2, 2016

That segfault looks like the one I got; it was fixed by upgrading LLVM to 3.9.

@schovi

This comment has been minimized.

Show comment
Hide comment
@schovi

schovi Nov 3, 2016

Contributor

Hi, I am super exited about running Crystal on Raspi. I am trying to do Docker playground.

FROM debian

##########
# Install utils
RUN apt-get update && \
    apt-get install -y \
      git \
      make \
      dh-autoreconf \
      apt-transport-https

##########
# Install LLVM
RUN echo "deb http://apt.llvm.org/trusty/ llvm-toolchain-trusty-3.9 main" >> /etc/apt/sources.list && \
    echo "deb-src http://apt.llvm.org/trusty/ llvm-toolchain-trusty-3.9 main" >> /etc/apt/sources.list

RUN apt-get update && \
    apt-get install -y --force-yes clang-3.9 lldb-3.9

##########
# Install other necessary libraries
RUN apt-get update && \
    apt-get install -y \
      libbsd-dev \
      libedit-dev \
      libevent-core-2.0-5 \
      libevent-dev \
      libevent-extra-2.0-5 \
      libevent-openssl-2.0-5 \
      libevent-pthreads-2.0-5 \
      libgmp-dev \
      libgmpxx4ldbl \
      libssl-dev \
      libxml2-dev \
      libyaml-dev \
      libreadline-dev

# Install BOEHM GC
ADD https://api.github.com/repos/ivmai/bdwgc/compare/master...HEAD /dev/null
RUN git clone https://github.com/ivmai/bdwgc.git /tmp/bdwgc

ADD https://api.github.com/repos/ivmai/libatomic_ops/compare/master...HEAD /dev/null
RUN git clone https://github.com/ivmai/libatomic_ops.git /tmp/bdwgc/libatomic_ops

RUN cd /tmp/bdwgc && autoreconf -vif
RUN cd /tmp/bdwgc && automake --add-missing
RUN cd /tmp/bdwgc && ./configure
RUN cd /tmp/bdwgc && make
RUN cd /tmp/bdwgc && make check
RUN cd /tmp/bdwgc && make install

##########
# Install crystal
RUN apt-key adv --keyserver keys.gnupg.net --recv-keys 09617FD37CC06B54 && \
    echo "deb https://dist.crystal-lang.org/apt crystal main" > /etc/apt/sources.list.d/crystal.list

RUN apt-get update && \
    apt-get install -y crystal

##########
# Add crystal repo
ADD https://api.github.com/repos/crystal-lang/crystal/compare/master...HEAD /dev/null
RUN git clone https://github.com/crystal-lang/crystal.git /tmp/crystal

##########
# Compile it
RUN cd /tmp/crystal && make libcrystal

RUN cd /tmp/crystal && \
    bin/crystal build \
      src/compiler/crystal.cr \
      --cross-compile \
      --target arm-unknown-linux-gnueabihf \
      --release -Dwithout_openssl -Dwithout_zlib

CMD ["sleep", "3600"]

Maybe it will help to somebody. When you have docker installed and running, just save this file as Dockerfile anywhere and then in same directory run docker build . -t crystal

Unfortunately it is not 100% yet and there is few problems. Like installing Boehm GC 7.6 throws error when calling: ./configure

...
checking sys/dg_sys_info.h presence... no
checking for sys/dg_sys_info.h... no
./configure: line 16943: syntax error near unexpected token `ATOMIC_OPS,'
./configure: line 16943: `   PKG_CHECK_MODULES(ATOMIC_OPS, atomic_ops, ,'
The command '/bin/sh -c cd /tmp/bdwgc && ./configure' returned a non-zero code: 2
Contributor

schovi commented Nov 3, 2016

Hi, I am super exited about running Crystal on Raspi. I am trying to do Docker playground.

FROM debian

##########
# Install utils
RUN apt-get update && \
    apt-get install -y \
      git \
      make \
      dh-autoreconf \
      apt-transport-https

##########
# Install LLVM
RUN echo "deb http://apt.llvm.org/trusty/ llvm-toolchain-trusty-3.9 main" >> /etc/apt/sources.list && \
    echo "deb-src http://apt.llvm.org/trusty/ llvm-toolchain-trusty-3.9 main" >> /etc/apt/sources.list

RUN apt-get update && \
    apt-get install -y --force-yes clang-3.9 lldb-3.9

##########
# Install other necessary libraries
RUN apt-get update && \
    apt-get install -y \
      libbsd-dev \
      libedit-dev \
      libevent-core-2.0-5 \
      libevent-dev \
      libevent-extra-2.0-5 \
      libevent-openssl-2.0-5 \
      libevent-pthreads-2.0-5 \
      libgmp-dev \
      libgmpxx4ldbl \
      libssl-dev \
      libxml2-dev \
      libyaml-dev \
      libreadline-dev

# Install BOEHM GC
ADD https://api.github.com/repos/ivmai/bdwgc/compare/master...HEAD /dev/null
RUN git clone https://github.com/ivmai/bdwgc.git /tmp/bdwgc

ADD https://api.github.com/repos/ivmai/libatomic_ops/compare/master...HEAD /dev/null
RUN git clone https://github.com/ivmai/libatomic_ops.git /tmp/bdwgc/libatomic_ops

RUN cd /tmp/bdwgc && autoreconf -vif
RUN cd /tmp/bdwgc && automake --add-missing
RUN cd /tmp/bdwgc && ./configure
RUN cd /tmp/bdwgc && make
RUN cd /tmp/bdwgc && make check
RUN cd /tmp/bdwgc && make install

##########
# Install crystal
RUN apt-key adv --keyserver keys.gnupg.net --recv-keys 09617FD37CC06B54 && \
    echo "deb https://dist.crystal-lang.org/apt crystal main" > /etc/apt/sources.list.d/crystal.list

RUN apt-get update && \
    apt-get install -y crystal

##########
# Add crystal repo
ADD https://api.github.com/repos/crystal-lang/crystal/compare/master...HEAD /dev/null
RUN git clone https://github.com/crystal-lang/crystal.git /tmp/crystal

##########
# Compile it
RUN cd /tmp/crystal && make libcrystal

RUN cd /tmp/crystal && \
    bin/crystal build \
      src/compiler/crystal.cr \
      --cross-compile \
      --target arm-unknown-linux-gnueabihf \
      --release -Dwithout_openssl -Dwithout_zlib

CMD ["sleep", "3600"]

Maybe it will help to somebody. When you have docker installed and running, just save this file as Dockerfile anywhere and then in same directory run docker build . -t crystal

Unfortunately it is not 100% yet and there is few problems. Like installing Boehm GC 7.6 throws error when calling: ./configure

...
checking sys/dg_sys_info.h presence... no
checking for sys/dg_sys_info.h... no
./configure: line 16943: syntax error near unexpected token `ATOMIC_OPS,'
./configure: line 16943: `   PKG_CHECK_MODULES(ATOMIC_OPS, atomic_ops, ,'
The command '/bin/sh -c cd /tmp/bdwgc && ./configure' returned a non-zero code: 2
@Sija

This comment has been minimized.

Show comment
Hide comment
@Sija

Sija Nov 3, 2016

Contributor

@schovi Did you install autotools (as per ivmai/bdwgc#69 (comment))?

Contributor

Sija commented Nov 3, 2016

@schovi Did you install autotools (as per ivmai/bdwgc#69 (comment))?

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Nov 3, 2016

Member

@schovi just use gc 7.6.0 and libatomic_ops 4.4.0 tarballs, instead of cloning the Git master.

Member

ysbaddaden commented Nov 3, 2016

@schovi just use gc 7.6.0 and libatomic_ops 4.4.0 tarballs, instead of cloning the Git master.

@rdp

This comment has been minimized.

Show comment
Hide comment
@rdp

rdp Nov 29, 2016

Contributor

@schovi ran into that as well, seems the work around was to install and run 'pkg-config' once there was a note added to the README recently about it I think. GL!

Contributor

rdp commented Nov 29, 2016

@schovi ran into that as well, seems the work around was to install and run 'pkg-config' once there was a note added to the README recently about it I think. GL!

@elorest

This comment has been minimized.

Show comment
Hide comment
@elorest

elorest Mar 16, 2017

Contributor

When trying to use the version of crystal I compiled on my raspi I get the following error when trying to compile a simple hello world app. Error in hello.cr:1: while requiring "prelude": can't find file 'prelude' relative to '/home/pi/crystal' I've been using crystal for a couple years on OSX and Linux (x86) and I've never seen this before. What did I do wrong?

Contributor

elorest commented Mar 16, 2017

When trying to use the version of crystal I compiled on my raspi I get the following error when trying to compile a simple hello world app. Error in hello.cr:1: while requiring "prelude": can't find file 'prelude' relative to '/home/pi/crystal' I've been using crystal for a couple years on OSX and Linux (x86) and I've never seen this before. What did I do wrong?

@lbguilherme

This comment has been minimized.

Show comment
Hide comment
@lbguilherme

lbguilherme Mar 16, 2017

Contributor

Is the prelude.cr file there? Everything in src/ must be available to the compiler at runtime.

Contributor

lbguilherme commented Mar 16, 2017

Is the prelude.cr file there? Everything in src/ must be available to the compiler at runtime.

@elorest

This comment has been minimized.

Show comment
Hide comment
@elorest

elorest Mar 16, 2017

Contributor
Contributor

elorest commented Mar 16, 2017

@elorest

This comment has been minimized.

Show comment
Hide comment
@elorest

elorest Mar 16, 2017

Contributor

I moved my hello.cr into crystal/src which I verified did have prelude.cr. When I then ran pi@raspberrypi:~/crystal/src $ ../bin/crystal build hello.cr
but still got this error:
Error in hello.cr:1: while requiring "prelude": can't find file 'prelude' relative to '/home/pi/crystal/src'

Contributor

elorest commented Mar 16, 2017

I moved my hello.cr into crystal/src which I verified did have prelude.cr. When I then ran pi@raspberrypi:~/crystal/src $ ../bin/crystal build hello.cr
but still got this error:
Error in hello.cr:1: while requiring "prelude": can't find file 'prelude' relative to '/home/pi/crystal/src'

@RX14

This comment has been minimized.

Show comment
Hide comment
@RX14

RX14 Mar 16, 2017

Member

You need to set CRYSTAL_PATH=lib:/path/to/crystal/source/src. I've updated my stackoverflow answer here.

Member

RX14 commented Mar 16, 2017

You need to set CRYSTAL_PATH=lib:/path/to/crystal/source/src. I've updated my stackoverflow answer here.

@elorest

This comment has been minimized.

Show comment
Hide comment
@elorest

elorest Mar 16, 2017

Contributor

@ysbaddaden I finally got crystal working on my raspberry pi. 👍 When I try to compile with it though I get this error https://dl.dropboxusercontent.com/s/m5mvghbb8q043be/2017-03-16%20at%2012.19%20PM.png

@RX14 Advised me to change crystal build hello.cr to crystal build --target arm-unknown-linux-gnueabihf hello.cr which worked. He also said that it was a bug and I should report it here. Thanks.

Contributor

elorest commented Mar 16, 2017

@ysbaddaden I finally got crystal working on my raspberry pi. 👍 When I try to compile with it though I get this error https://dl.dropboxusercontent.com/s/m5mvghbb8q043be/2017-03-16%20at%2012.19%20PM.png

@RX14 Advised me to change crystal build hello.cr to crystal build --target arm-unknown-linux-gnueabihf hello.cr which worked. He also said that it was a bug and I should report it here. Thanks.

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Mar 17, 2017

Member

@elorest you copied the binary over to the pi, right? The compiler needs a bash script that sets CRYSTAL_PATH and eventually delegates to the binary —see any Linux or macOS distribution.

Default target issue: how did you install or compile LLVM for the rpi? Crystal asks LLVM what the default host target is. You can check with llvm-config --host-target; it should be arm-unknown-linux-gnueabihf but I guess it's not?

Member

ysbaddaden commented Mar 17, 2017

@elorest you copied the binary over to the pi, right? The compiler needs a bash script that sets CRYSTAL_PATH and eventually delegates to the binary —see any Linux or macOS distribution.

Default target issue: how did you install or compile LLVM for the rpi? Crystal asks LLVM what the default host target is. You can check with llvm-config --host-target; it should be arm-unknown-linux-gnueabihf but I guess it's not?

@RX14

This comment has been minimized.

Show comment
Hide comment
@RX14

RX14 Mar 17, 2017

Member

@ysbaddaden I got @elorest to fix the crystal path issue by setting CRYSTAL_CONFIG_PATH.

Member

RX14 commented Mar 17, 2017

@ysbaddaden I got @elorest to fix the crystal path issue by setting CRYSTAL_CONFIG_PATH.

@elorest

This comment has been minimized.

Show comment
Hide comment
@elorest

elorest Mar 18, 2017

Contributor

I installed in from a debian repo that i needed to add since the newest stable version was 3.5. It installed it as llvm-config-3.9 which I specified when I compiled on my arm machine from cyrstal.o.
It returns armv7l-unknown-linux-gnueabihf from --host-target.

Contributor

elorest commented Mar 18, 2017

I installed in from a debian repo that i needed to add since the newest stable version was 3.5. It installed it as llvm-config-3.9 which I specified when I compiled on my arm machine from cyrstal.o.
It returns armv7l-unknown-linux-gnueabihf from --host-target.

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Mar 18, 2017

Member

The issue may be that Crystal doesn't deal correctly with the armv7l part, and searches for a lib_c/armv7l-linux-gnueabihf which I'm not really sure how to fix. Maybe force it to arm-....

Member

ysbaddaden commented Mar 18, 2017

The issue may be that Crystal doesn't deal correctly with the armv7l part, and searches for a lib_c/armv7l-linux-gnueabihf which I'm not really sure how to fix. Maybe force it to arm-....

@ysbaddaden

This comment has been minimized.

Show comment
Hide comment
@ysbaddaden

ysbaddaden Mar 20, 2017

Member

@elorest Does the fix in #4167 fixes the target issue? are Crystal specs passing on your RPI?

Member

ysbaddaden commented Mar 20, 2017

@elorest Does the fix in #4167 fixes the target issue? are Crystal specs passing on your RPI?

ysbaddaden added a commit to ysbaddaden/crystal that referenced this pull request Mar 23, 2017

Fix: select generic ARM/AArch64 LibC
Fixes the target triple used by Crystal to select the LibC bindings
when the triple specifies a specific ARM architecture, e.g. `armv6`,
`armv7l` or `armv8.1-a`.

refs crystal-lang#3424 (comment)

ysbaddaden added a commit that referenced this pull request Mar 23, 2017

Fix: select generic ARM/AArch64 LibC (#4167)
Fixes the target triple used by Crystal to select the LibC bindings
when the triple specifies a specific ARM architecture, e.g. `armv6`,
`armv7l` or `armv8.1-a`.

refs #3424 (comment)
@elorest

This comment has been minimized.

Show comment
Hide comment
@elorest

elorest Mar 23, 2017

Contributor

@ysbaddaden Yes they pass when I pass arm-unknown-linux-gnueabih as the target. I've since messed something up on the raspberry pi system I was using so can't try that latest fix yet. I'll let you know when I check that.

Contributor

elorest commented Mar 23, 2017

@ysbaddaden Yes they pass when I pass arm-unknown-linux-gnueabih as the target. I've since messed something up on the raspberry pi system I was using so can't try that latest fix yet. I'll let you know when I check that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment