New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add support for arm-linux-musleabihf #5862

Open
wants to merge 2 commits into
base: master
from

Conversation

Projects
None yet
4 participants
@jirutka
Copy link
Contributor

jirutka commented Mar 25, 2018

⚠️ There’s some problem, see comments…

Related to #5467

@jirutka jirutka force-pushed the jirutka:arm-linux-musleabihf branch 2 times, most recently from f770931 to 794525b Mar 25, 2018

@@ -36,9 +36,11 @@ class Crystal::Program
set.add "x86_64" if set.any?(&.starts_with?("amd64"))
set.add "i686" if set.any? { |flag| %w(i586 i486 i386).includes?(flag) }

set.add "musl" if set.includes?("musleabihf")

This comment has been minimized.

@RX14

RX14 Mar 25, 2018

Member

you should probably add "gnu" if it includes gnueabihf, just to make this symmetrical. It's not currently used but it may be in the future.

@RX14

This comment has been minimized.

Copy link
Member

RX14 commented Mar 25, 2018

Same as #5861: if you can cross-compile the compiler and run the spec suite sucesfully on the target machine this is good to go.

@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 25, 2018

if you can cross-compile the compiler and run the spec suite sucesfully on the target machine this is good to go.

I did not yet, it’s in progress. ;)

@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 25, 2018

It seems that I did something wrong, compiling crystal on armhf with cross-compiled crystal fails:

$ ./bin/crystal build --release --progress --threads 8 --static  --target armv6-alpine-linux-musleabihf  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Invalid memory access (signal 7) at address 0xab70227b
[0xab6cfbf0] __crystal_sigfault_handler +8865944
[0xaae5b388] __crystal_sigfault_handler +48
make: *** [Makefile:118: .build/crystal] Error 7
$ strace ./bin/crystal build --release --progress --threads 8 --static  --target armv6-alpine-linux-musleabihf  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
...
access("/home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/ecr/./lexer.cr", F_OK) = 0
open("/home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/ecr/lexer.cr", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 8
fcntl64(8, F_SETFD, FD_CLOEXEC)         = 0
fstat64(8, {st_mode=S_IFREG|0644, st_size=4016, ...}) = 0
read(8, "# :nodoc:\nclass ECR::Lexer\n  cla"..., 4096) = 4016
read(8, "", 4096)                       = 0
close(8)                                = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=746226671}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=746297972}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=749744730}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=749847751}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=796043327}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=796122868}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=803931975}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=804027436}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=805860037}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=805948298}) = 0
brk(0xabd2c000)                         = 0xabd2c000
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=838290639}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712533, tv_nsec=838452660}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=113001805}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=113125246}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=115341491}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=115441232}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=119568498}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=45712534, tv_nsec=119682340}) = 0
brk(0xabd2d000)                         = 0xabd2d000
brk(0xabd2f000)                         = 0xabd2f000
brk(0xabd30000)                         = 0xabd30000
brk(0xabd31000)                         = 0xabd31000
brk(0xabd32000)                         = 0xabd32000
--- SIGBUS {si_signo=SIGBUS, si_code=BUS_ADRALN, si_addr=0xab5ee27b} ---
fcntl64(0, F_GETFL)                     = 0x20402 (flags O_RDWR|O_APPEND|O_LARGEFILE)
fcntl64(0, F_SETFL, O_RDWR|O_APPEND|O_NONBLOCK|O_LARGEFILE) = 0
fcntl64(1, F_GETFL)                     = 0x20c02 (flags O_RDWR|O_APPEND|O_NONBLOCK|O_LARGEFILE)
fcntl64(2, F_GETFL)                     = 0x20c02 (flags O_RDWR|O_APPEND|O_NONBLOCK|O_LARGEFILE)
writev(2, [{iov_base="Invalid memory access (signal 7)"..., iov_len=55}, {iov_base=NULL, iov_len=0}], 2Invalid memory access (signal 7) at address 0xab5ee27b
) = 55
writev(2, [{iov_base="[0xab5bbbf0] __crystal_sigfault_"..., iov_len=49}, {iov_base=NULL, iov_len=0}], 2[0xab5bbbf0] __crystal_sigfault_handler +8865944
) = 49
writev(2, [{iov_base="[0xaad47388] __crystal_sigfault_"..., iov_len=44}, {iov_base=NULL, iov_len=0}], 2[0xaad47388] __crystal_sigfault_handler +48
) = 44
exit_group(7)                           = ?
@codenoid
Copy link
Contributor

codenoid left a comment

you need to fix

@RX14

This comment has been minimized.

Copy link
Member

RX14 commented Mar 25, 2018

@jirutka can you run in gdb and show a backtrace when it segfaults?

@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 25, 2018

(gdb) run build --release --progress --threads 8  --target armv6-alpine-linux-musleabihf  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Starting program: /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2-armv6-alpine-linux-musleabihf/bin/crystal build --release --progress --threads 8  --target armv6-alpine-linux-musleabihf  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
[2/13] Semantic (top level)
Program received signal SIGBUS, Bus error.
__crystal_personality () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/raise.cr:40
40	{% if flag?(:arm) %}

(gdb) backtrace
#0  __crystal_personality () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/raise.cr:40
#1  0xf538a3a4 in ?? () from /usr/lib/libgcc_s.so.1
#2  0xf538aec0 in _Unwind_RaiseException () from /usr/lib/libgcc_s.so.1
#3  0xaaab653c in __crystal_raise () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/raise.cr:167
#4  0xaac2f518 in raise () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/raise.cr:189
#5  0xaade2334 in raise () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/ast.cr:13
#6  raise:exception_type () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/ast.cr:12
#7  interpret_raise () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/methods.cr:169
#8  interpret_top_level_call? () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/methods.cr:32
#9  0xaadd8d1c in interpret_top_level_call () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/methods.cr:8
#10 visit () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/interpreter.cr:359
#11 accept () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/syntax/visitor.cr:27
#12 0xaadd9824 in visit () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/interpreter.cr:334
#13 accept () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/syntax/visitor.cr:27
#14 0xaaf0459c in visit () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/interpreter.cr:99
#15 accept () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/syntax/visitor.cr:27
#16 expand_macro () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/macros/macros.cr:48
#17 expand_inline_macro () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/semantic_visitor.cr:386
#18 0xaaeab2c0 in expand_inline_macro () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/semantic_visitor.cr:372
#19 visit () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/semantic_visitor.cr:155
#20 accept () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/syntax/visitor.cr:27
#21 0xaaf01bc8 in visit () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic/top_level_visitor.cr:619
#22 0xaaeaae04 in accept () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/syntax/visitor.cr:27
#23 0xaaea729c in top_level_semantic () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic.cr:62
#24 0xaaea4ca4 in semantic () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/semantic.cr:22
#25 0xab2d5dec in compile () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/compiler.cr:144
#26 0xaaaee0dc in compile () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/command.cr:265
#27 compile () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/command.cr:263
#28 build () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/command.cr:165
#29 run () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/command.cr:70
#30 0xaaab5e84 in __crystal_main () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/compiler/crystal/command.cr:49
#31 0xaaabbd38 in _crystal_main () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/crystal/main.cr:11
#32 main_user_code () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/crystal/main.cr:112
#33 main () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/crystal/main.cr:101
#34 main () at /home/jirutjak/aports/testing/crystal/src/crystal-0.24.2/src/crystal/main.cr:135

(I’m building crystal 0.24.2 with patch for armhf and patch for aarch64 support)

@RX14

This comment has been minimized.

Copy link
Member

RX14 commented Mar 25, 2018

Hmm, looks like raising/libunwind is different on musl arm. They don't seem to be different on /x86(_64)?/ musl.

jirutka added some commits Mar 25, 2018

Add compiler flag "gnu" when "gnueabihf" is present
To be symetrical with musl. It's not currently used but it may be in
the future.

@jirutka jirutka force-pushed the jirutka:arm-linux-musleabihf branch from 01be976 to 3236720 Mar 25, 2018

@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 25, 2018

I’ve tried to cross-compile for armhf with -fno-omit-frame-pointer (we use -fomit-frame-pointer on Alpine by default) and used it to successfully built crystal on armhf. However, I’ve also built it in non-release mode now, to avoid waiting so long, so I’m now not sure what actually helped.

@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 25, 2018

Oh crap, it worked thanks to non-release build, not -fno-omit-frame-pointer. 😿

Failed to raise an exception: FAILURE
[0x910e88] ???
[0x76898] ???
[0x77a34] ???
[0x91126c] ???
[0x9c600] ???
make: *** [Makefile:118: .build/crystal] Error 9

Hm, what --release actually do? It probably doesn’t just increase optimization level, does it?

@RX14

This comment has been minimized.

Copy link
Member

RX14 commented Mar 25, 2018

@jirutka yes, it's just compiler optimization. That's strange. Probably a compiler (or llvm) bug. We're still failing to raise/unwind anyway so it hasn't helped much...

@ysbaddaden

This comment has been minimized.

Copy link
Member

ysbaddaden commented Mar 25, 2018

Unwinding is specific on ARM EHABI architectures. I spent a lot of time to decipher it, and mostly based the implementation on how libgcc / libstdc++ handle this... But it may be different for musl.

See src/callstack/lib_unwind.cr, and there are closed ARM related issues in which I explained things as I found them.

There is also an opened issue with unwinding and static binaries on musl/alpine.

@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 25, 2018

@ysbaddaden Aha! The above error is from static binary (I’m building static crystal binaries to be used for bootstrapping normal builds), so I’ll try dynamic binary.

@ysbaddaden

This comment has been minimized.

Copy link
Member

ysbaddaden commented Mar 25, 2018

You can build glibc static binaries to help bootstrapping. There will be the usual warnings, but meh, the binary will be good enough for bootstrap a native build —this is how I initially added support for alpine x86.

@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 26, 2018

It seems that even dynamically linked does not work. :(

Building crystal with dynamically linked crystal built on armv6 machine:

$ ./bin/crystal build --release --progress --threads 8  --verbose --target armv6-alpine-linux-musleabihf  --stats -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Parse:                             00:00:00.000168842 (   0.25MB)
Semantic (top level):              00:00:01.578067935 (  50.77MB)
Semantic (new):                    00:00:00.010198093 (  50.77MB)
Semantic (type declarations):      00:00:00.205687239 (  58.77MB)
Semantic (abstract def check):     00:00:00.009524184 (  58.77MB)
Semantic (ivars initializers):     00:00:00.529359294 (  58.80MB)
Semantic (cvars initializers):     00:00:00.012355161 (  58.80MB)
Invalid memory access (signal 7) at address 0xab4db28b
[0xab4a8bf0] __crystal_sigfault_handler +8865944
[0xaac34388] __crystal_sigfault_handler +48
make: *** [Makefile:118: .build/crystal] Error 7
@ysbaddaden

This comment has been minimized.

Copy link
Member

ysbaddaden commented Mar 26, 2018

You may look for the :gnueabihf flag. When present we set the :armhf flag that will eventually enable a conservative default hardware floating point unit, otherwise LLVM defaults to a software floating point, which can crash with C bindings, because they expect fp registers but LLVM uses int registers (and vice versa).

If it still fails, you may investigate by cross building each spec and running it one by one on the ARM target —with an automated script— and find what's crashing.

Supporting ARM32 isn't an easy feat...

@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 28, 2018

I wanted to give you access to ARMv7 machine, so you can troubleshoot it, but… it seems that the same binary (built on another ARM machine) does not crash here! 😕 It’s still running, but already passed the time when it repeatedly crashed on the another ARM machine.

The machine I use is actually ARMv8, but it is capable running in 32 bit mode. The new one is VM on Scaleway, real ARMv7 machine.

@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 29, 2018

Okay, I’ve successfully built dynamically linked crystal (3) on musl ARMv7 using crystal (2) compiled on the same machine using crystal (1) cross-compiled from x86_64 musl. I also run specs with the following result:

Finished in 20:29 minutes
11444 examples, 24 failures, 5 errors, 11 pending

Failed examples:

crystal spec spec/compiler/codegen/and_spec.cr:32 # Code gen: and codegens and with primitive type other than bool with union
crystal spec spec/compiler/codegen/c_struct_spec.cr:10 # Code gen: struct codegens struct property setter
crystal spec spec/compiler/codegen/c_struct_spec.cr:14 # Code gen: struct codegens struct property setter via pointer
crystal spec spec/compiler/codegen/c_struct_spec.cr:18 # Code gen: struct codegens struct property setter via pointer
crystal spec spec/compiler/codegen/c_union_spec.cr:10 # Code gen: c union codegens union property default value 2
crystal spec spec/compiler/codegen/c_union_spec.cr:18 # Code gen: c union codegens union property setter 2
crystal spec spec/compiler/codegen/c_union_spec.cr:26 # Code gen: c union codegens union property setter 2 via pointer
crystal spec spec/compiler/codegen/class_spec.cr:48 # Code gen: class codegens method call of instance var
crystal spec spec/compiler/codegen/const_spec.cr:26 # Codegen: const finds nearest constant first
crystal spec spec/compiler/codegen/const_spec.cr:42 # Codegen: const allows constants with same name
crystal spec spec/compiler/codegen/if_spec.cr:32 # Code gen: if codegens if with union
crystal spec spec/compiler/codegen/or_spec.cr:32 # Code gen: or codegens or with primitive type other than bool with union
crystal spec spec/compiler/codegen/primitives_spec.cr:45 # Code gen: primitives codegens f32
crystal spec spec/compiler/codegen/primitives_spec.cr:49 # Code gen: primitives codegens f64
crystal spec spec/compiler/codegen/union_type_spec.cr:4 # Code gen: union type codegens union type when obj is union and no args
crystal spec spec/compiler/codegen/union_type_spec.cr:8 # Code gen: union type codegens union type when obj is union and arg is union
crystal spec spec/compiler/codegen/union_type_spec.cr:12 # Code gen: union type codegens union type when obj is not union but arg is
crystal spec spec/compiler/codegen/union_type_spec.cr:16 # Code gen: union type codegens union type when obj union but arg is not
crystal spec spec/compiler/codegen/union_type_spec.cr:20 # Code gen: union type codegens union type when no obj
crystal spec spec/compiler/codegen/union_type_spec.cr:24 # Code gen: union type codegens union type when no obj and restrictions
crystal spec spec/compiler/codegen/union_type_spec.cr:28 # Code gen: union type codegens union type as return value
crystal spec spec/compiler/codegen/union_type_spec.cr:32 # Code gen: union type codegens union type for instance var
crystal spec spec/std/crc32_spec.cr:10 # CRC32 should be able to calculate crc32 combined
crystal spec spec/std/socket_spec.cr:416 # TCPServer fails when port is in use
crystal spec spec/std/socket_spec.cr:451 # TCPSocket sends and receives messages
crystal spec spec/std/socket_spec.cr:530 # TCPSocket fails when connection is refused
crystal spec spec/std/socket_spec.cr:562 # UDPSocket reads and writes data to server
crystal spec spec/std/socket_spec.cr:582 # UDPSocket sends and receives messages over IPv4
crystal spec spec/std/socket_spec.cr:613 # UDPSocket sends and receives messages over IPv6

More details for two randomly selected tests reveals something interesting:

  1) Code gen: struct codegens struct property setter
     Failure/Error: run("#{CodeGenStructString}; bar = LibFoo::Bar.new; bar.y = 2.5_f32; bar.y").to_f32.should eq(2.5)

       Expected: 2.5
            got: 7.006492321624085e-45

     # spec/compiler/codegen/c_struct_spec.cr:11

  1) Code gen: c union codegens union property default value 2
     Failure/Error: run("#{CodeGenUnionString}; bar = Pointer(LibFoo::Bar).malloc(1_u64); bar.value.z").to_f32.should eq(0)

       Expected: 0
            got: 7.006492321624085e-45

     # spec/compiler/codegen/c_union_spec.cr:11

Does these tests pass on glibc armhf? I have an impression that this may not be related to musl bindings at all and it looks like these failing tests and SIGBUS on ARMv8 in 32bit mode may be two different problems.

And there’s log for the first failing socket test, it hangs on it:

HTTP::WebSocket
...
  close
    closes with message
    closes without message
  negotiates over HTTP correctlyUnhandled exception in spawn:
bind: Address not available (Errno)
  from src/socket/tcp_server.cr:71:15 in 'initialize'
  from src/socket/tcp_server.cr:32:3 in 'initialize:reuse_port'
  from TCPServer::new:reuse_port<String, Int32, Bool>:TCPServer
  from src/http/server.cr:154:5 in 'bind'
  from src/http/server.cr:153:3 in 'bind'
  from src/spec/expectations.cr:210:7 in '~procProc(Nil)'
  from src/fiber.cr:255:3 in 'run'
  from src/concurrent.cr:61:3 in '~proc2Proc(Fiber, (IO::FileDescriptor | Nil))'
@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 29, 2018

require "./src/compiler/crystal/**"
require "primitives"

include Crystal

code = <<-CODE
  lib LibFoo
    struct Bar
      x : Int32
      y : Float32
    end
  end

  bar = LibFoo::Bar.new
  bar.y = 2.5_f32
  bar.y
CODE

x = Program.new.run(code, debug: Crystal::Debug::None).to_f32
puts x == 2.5
puts x
$ crystal build foo.rs
$ DUMP=1 ./foo

; ModuleID = 'main_module'
source_filename = "main_module"
target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64"

%"struct.LibFoo::Bar" = type { i32, float }

@ARGC_UNSAFE = internal global i32 0
@ARGV_UNSAFE = internal global i8** null

define float @__crystal_main(i32 %argc, i8** %argv) {
alloca:
  %bar = alloca %"struct.LibFoo::Bar"
  %0 = alloca %"struct.LibFoo::Bar"
  br label %entry

entry:                                            ; preds = %alloca
  store i32 %argc, i32* @ARGC_UNSAFE
  store i8** %argv, i8*** @ARGV_UNSAFE
  %1 = call %"struct.LibFoo::Bar" @"*struct.LibFoo::Bar::new:struct.LibFoo::Bar"()
  store %"struct.LibFoo::Bar" %1, %"struct.LibFoo::Bar"* %0
  %2 = load %"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* %0
  store %"struct.LibFoo::Bar" %2, %"struct.LibFoo::Bar"* %bar
  %3 = getelementptr inbounds %"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* %bar, i32 0, i32 1
  store float 2.500000e+00, float* %3
  %4 = getelementptr inbounds %"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* %bar, i32 0, i32 1
  %5 = load float, float* %4
  ret float %5
}

declare i32 @printf(i8*, ...)

; Function Attrs: uwtable
define internal %"struct.LibFoo::Bar" @"*struct.LibFoo::Bar::new:struct.LibFoo::Bar"() #0 {
alloca:
  %x = alloca %"struct.LibFoo::Bar"
  %0 = alloca %"struct.LibFoo::Bar"
  br label %entry

entry:                                            ; preds = %alloca
  %1 = bitcast %"struct.LibFoo::Bar"* %0 to i8*
  call void @llvm.memset.p0i8.i32(i8* %1, i8 0, i32 ptrtoint (%"struct.LibFoo::Bar"* getelementptr (%"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* null, i32 1) to i32), i32 4, i1 false)
  %2 = load %"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* %0
  store %"struct.LibFoo::Bar" %2, %"struct.LibFoo::Bar"* %x
  %3 = load %"struct.LibFoo::Bar", %"struct.LibFoo::Bar"* %x
  ret %"struct.LibFoo::Bar" %3
}

; Function Attrs: argmemonly nounwind
declare void @llvm.memset.p0i8.i32(i8* nocapture writeonly, i8, i32, i32, i1) #1

attributes #0 = { uwtable }
attributes #1 = { argmemonly nounwind }
false
7.006492321624085e-45
@ysbaddaden

This comment has been minimized.

Copy link
Member

ysbaddaden commented Mar 29, 2018

Except for the Target DataLayout, the LLVM IR dump is identical on x86_64 (that was expected). It looks more and more related to the hardware, and possibly related to the FPU, maybe a software vs hardware expectations between libraries.

  • Is a hardware FPU enabled? Since +vfp2 is set by default for the (gnu|musl)eabihf targets, I assume it does;
  • Maybe tweak --mcpu and --mattr to closely match the CPU/FPU?

Can the program itself print the FP value correctly?

  • no => possible LLVM code generation issue (or FPU mismatch);
  • yes => issue in LLVM GenericValue when it tries to access the evaluated Float32 value in memory: it's not where it's expected or invalid; maybe LLVM JitCompiler used the wrong register to get the value from to store it in memory —which would explain why the value is the same across different computations.
@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 29, 2018

lib LibFoo
  struct Bar
    x : Int32
    y : Float32
  end
end

bar = LibFoo::Bar.new
bar.y = 2.5_f32
x = bar.y

puts x.to_f32 == 2.5
puts x
puts x / 2
$ crystal run bar.rs
true
2.5
1.25
@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Mar 29, 2018

@ysbaddaden I’ve prepared you environment on baremetal ARMv7 machine to troubleshoot this issue: ysbaddaden@163.172.130.162 (login with your SSH key).

@ysbaddaden

This comment has been minimized.

Copy link
Member

ysbaddaden commented Mar 30, 2018

Thanks! I'll try to have a look over the weekend.

I confirm the issue: the LLVMGenericValue doesn't get the value returned by the Crystal module, but from some other place. I had this issue with Crystal expecting the value in a regular CPU registers when a library actually put it in a VFP register.

@jirutka

This comment has been minimized.

Copy link
Contributor

jirutka commented Apr 2, 2018

I'll try to have a look over the weekend.

How it goes? :)

@ysbaddaden

This comment has been minimized.

Copy link
Member

ysbaddaden commented Apr 3, 2018

Sorry, I didn't have much computer time. I'm not sure I'll have until a while :(

@jirutka jirutka referenced this pull request May 7, 2018

Closed

Builds for armhf and aarch64 #6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment