FreeBSD/arm support

[rpaulo]

These changes enable compilation on FreeBSD/arm. The ARM JIT compiler doesn't work on this platform (it crashes, I did not investigate).

racket/racket/bin % ./racket   
Welcome to Racket v6.2.900.5.
> (displayln "hello world")
hello world
> ^D
racket/racket/bin % uname -srm
FreeBSD 11.0-CURRENT arm

[rpaulo]

@mflatt I was told you might want to take a look at this PR. Thanks.

[mflatt]

I've merged this change, but I'd like to leave the PR open until I can investigate the JIT crash.

Does the build crash attempting to run racketcgc (to build racket3m) or only later?

[rpaulo]

When running racketcgc. I did not gather any debugging information yet...

[mflatt]

My attempts to run FreeBSD on an ARM machine (Raspberry Pi) haven't worked out, so far. In case the problem is easy to track down, can you try starting racketcgc in gdb? I expect that it will crash quickly, and the stack trace combined with disassembly near the crash might make the problem clear.

[rpaulo]

The Raspberry Pi probably doesn't have enough RAM (depending on the model). What problems did you have? I made a mistake and my BeagleBone is offline now, but I will try to debug this once it's back up.

[mflatt]

I couldn't get it to boot after dding an image to an SD card. The boot failed with "unrecognized filesystem type" – which probably reflects some really basic problem, but searching for solutions didn't get me anywhere.

[rpaulo]

Are you using an image from ftp.freebsd.org? It was suggested on IRC that the SD card might be failing. The RPI is very picky about SD cards... Can you try a different SD card?

[mflatt]

Yes, I tried from ftp.freebsd.org. I tried a couple of SD cards, including a SanDisk (which I think I saw recommended somewhere), but that could still be it.

[rpaulo]

It seems like the JIT is generating an invalid instruction:

% gdb ../racketcgc  
(gdb) r
[New LWP 100112]
Welcome to Racket v6.2.900.5 [cgc].
[New Thread 20c03300 (LWP 100112/racketcgc)]

Program received signal SIGILL, Illegal instruction.
[Switching to Thread 20c03300 (LWP 100112/racketcgc)]
0x207e0014 in ?? ()
(gdb) 
#0  0x207e0014 in ?? ()
(gdb) 

[rpaulo]

These instructions don't make a lot of sense... More debugging tomorrow.

(gdb) x/10i 0x207e0000
0x207e0000:     andeq   r0, r0, r0
0x207e0004:     rsbscs  r0, lr, r0, lsr r0
0x207e0008:     rsbscs  pc, lr, r0, lsr r0
0x207e000c:     rsbscs  r0, lr, r0, lsr r6
0x207e0010:     andeq   r0, r0, r0, lsl #12
0x207e0014:     undefined instruction 0xffff0005
0x207e0018:     andeq   r0, r0, r0
0x207e001c:     cmpcs   r5, r0
0x207e0020:     andeq   r0, r0, r0
0x207e0024:     andeq   r0, r0, r0
(gdb) 

[mflatt]

You'll probably have to tell gdb explicitly to decode Thumb instructions with set arm force-mode thumb.

Since it's an illegal-instruction error, which may mean the JIT is generating the wrong instructions for your machine, what does gcc -dM -E - < /dev/null | grep ARM produce on that machine?

Thanks for your help!

[rpaulo]

Our gdb is quite old, so it doesn't have that option... Maybe you're right and it's a Thumb instruction that this gdb can't parse.

% cc -dM -E - < /dev/null | grep ARM  
#define __ARMEL__ 1
#define __ARM_32BIT_STATE 1
#define __ARM_ACLE 200
#define __ARM_ARCH 6
#define __ARM_ARCH_6ZK__ 1
#define __ARM_ARCH_ISA_ARM 1
#define __ARM_EABI__ 1
#define __ARM_FEATURE_DSP 1
#define __ARM_FP 0xC
#define __ARM_PCS 1
#define __ARM_SIZEOF_MINIMAL_ENUM 4
#define __ARM_SIZEOF_WCHAR_T 4
#define __ARM_VFPV2__ 1

[mflatt]

The preprocessor output says that Thumb is not supported, so the JIT shouldn't generate Thumb instructions, anyway.

I see that "racket/src/racket/src/lightning/arm/asm.h" doesn't specifically recognize __ARM_ARCH_6ZK__. Does it help to add that alongside __ARM_ARCH_6K__?

[rpaulo]

That didn't help. :(

[mflatt]

I got FreeBSD running on an RPI2. Unfortunately, the JIT seems to work in that configuration, so I'm still stuck on this problem.

[rpaulo]

How are you enabling the JIT? In sconfig.h? The processor on the BeagleBoard is similar to the one on the RPI2, so it's likely that I made a mistake.

[mflatt]

I added -DMZ_USE_JIT_ARM to CPPFLAGS, since I had to pass -I/usr/local/include to find the installed libffi, anyway:
../configure CPPFLAGS="-I/usr/local/include -DMZ_USE_JIT_I386" LDFLAGS=-L/usr/local/lib
I confirmed by running a loop that the JIT was enabled and that providing -j to racket made it go slower.

It should work the same to add the definition in "sconfig.h", though.

I had to install libffi, because the sources bundled with Racket didn't build. Did you install libffi via the package manager, too?

[rpaulo]

Yup, installed libffi and used roughly the same configure arguments as you did (you made a mistake in -DMZ_USE_JIT_I386, right?).

I used the port arguments: env CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib configure --disable-lt --enable-shared --enable-pthread

[rpaulo]

Can you email me your racketcgc binary?

[mflatt]

Good idea: http://www.cs.utah.edu/~mflatt/tmp/racketcgc

[rpaulo]

That crashes exactly the same way. Guess there's something special about this CPU...

[mflatt]

It's possible that the JIT is generating instructions that fall outside the spec, where different processors treat the undefined behavior in different ways. That has happened one time before. In that previous case, though, we were able to get disassembly from gdb to help track down the problem.

In this case, since the disassembly at the crash point doesn't make sense, it seems more likely that a jump is going wrong. Does a bt in gdb offer any further hints?

It's a long shot, but what if you change __START_SHORT_JUMPS__ in "jit.h" so that it doesn't change the long_jumps state?

[rpaulo]

A bit different:

% gdb ./racketcgc  
(gdb) r -cqu ./check-sdep.rkt
[New LWP 100110]
internal error in JIT;
 ending address 0x214a0adc not in [0x214a0630,0x214a0ad0] (0)
internal error: JIT buffer overflow
[New Thread 20c03300 (LWP 100110/racketcgc)]

Program received signal SIGABRT, Aborted.
[Switching to Thread 20c03300 (LWP 100110/racketcgc)]
0x204b0274 in thr_kill () from /lib/libc.so.7
(gdb) bt
#0  0x204b0274 in thr_kill () from /lib/libc.so.7
#1  0x204b0260 in raise () from /lib/libc.so.7
#2  0x204b01d8 in abort () from /lib/libc.so.7
#3  0x000bb710 in scheme_generate_one (old_jitter=<value optimized out>, 
    generate=<value optimized out>, data=<value optimized out>, gcable=<value optimized out>, 
    ndata=0x2070a128) at jitstate.c:401
#4  0x00087138 in scheme_on_demand_generate_lambda (nc=<value optimized out>, 
    argc=<value optimized out>, argv=<value optimized out>, argv_delta=<value optimized out>)
    at jit.c:3691
#5  0x000872f0 in scheme_on_demand (rs=0x211c5f90) at jit.c:3770
#6  0x203e0a94 in ?? ()
(gdb) 

However, this is the same crash:

% gdb ./racketcgc 
(gdb) r
[New LWP 100110]
Welcome to Racket v6.2.900.5 [cgc].
[New Thread 20c03300 (LWP 100110/racketcgc)]

Program received signal SIGILL, Illegal instruction.
[Switching to Thread 20c03300 (LWP 100110/racketcgc)]
0x207d0014 in ?? ()
(gdb) bt
#0  0x207d0014 in ?? ()
(gdb) 

[rpaulo]

I'm not working on this anymore. Shall we close?

[LiberalArtist]

The arrival of Racket CS seems like a good reason to close this.