Incorrect use of RIP Relative to access globals on x86_64

[llvmbot]

Bugzilla Link	1299
Resolution	WORKSFORME
Resolved on	Feb 22, 2010 12:43
Version	trunk
OS	All
Reporter	LLVM Bugzilla Contributor

Extended Description

Absolute addressing is never used to address a global variable. This is
incorrect as there is no guarantee that a given global is within 2GB of an
instruction referencing it. On some operating systems, such as FreeBSD, the JIT
will almost always find the globals more than 2GB away from code, but on any OS
all it takes is more than 2GB of global variables to make this happen.

This bug causes numerous JIT failures in llvm-test on FreeBSD.

[lattner]

Evan didn't you just fix this, by switching the JIT to large code model?

[llvmbot]

I tried explicitly specifying code models to lli and it had no effect. I also
looked at the code in Target/X86 and as far as I can tell it never uses absolute
addressing on x86_64; the only absolute addressing mode it can emit uses a
32-bit address and only for x86.

[lattner]

Evan, in your wealth of free time :), can you take a look at this when you get a chance?

[llvmbot]

JIT uses a (incomplete) large code model. You can emulate it with
-code-model=large. e.g.

extern int src[];
extern int dst[];

void test(void) {
dst[0] = src[0];
}

With llc -code-model=large, it generates:

    .align  4
    .globl  _test

_test:
movabsq $_dst, %rax
movq (%rax), %rax
movabsq $_src, %rcx
movq (%rcx), %rcx
movl (%rcx), %ecx
movl %ecx, (%rax)
ret

On Mac OS X X86-64, global variables can reside in high memory. The current
scheme works fine for Mac, but I have not tested it on any other systems. I
suspect it's failing on FreeBSD for other reasons. I am not at all familiar with
its ABI.

Jeff, can you add -print-machineinstrs or disassemble the machine instructions
generated by the JIT?

[llvmbot]

Running ProlangsC++/primes with "lli -debug", the tail end of the output shows:

JIT: Stub emitted at [0x8015fbff0] for function '_Znwm'
JIT: Stub emitted at [0x8015fbfe0] for function 'printf'
JIT: Finished CodeGen of [0x8015fc010] Function: main: 212 bytes of text, 9
relocations
Disassembled code:
No disassembler available. See configure help for options.
Global 'str1' -> 0xe178c0
Global 'str' -> 0xe17810
Global '_ZTV6filter' -> 0xe388e0
JIT: Stub emitted at [0x8015fbfd0] for function '_ZN6filter3outEv'
Global '_ZTI6filter' -> 0xe38660
Global '_ZTI4item' -> 0xe177c0
Global '_ZTS4item' -> 0xe177e0
Global '_ZTS6filter' -> 0xe176e0
Global '_ZTV7counter' -> 0xe38840
JIT: Stub emitted at [0x8015fbfc0] for function '_ZN7counter3outEv'
Global '_ZTI7counter' -> 0xe38720
Global '_ZTS7counter' -> 0xe17800
Segmentation fault (core dumped)

In the above, note the address of function main() and of global _ZTV7counter.
What follows is the machine code for function main up to the point where it crashes:

0x00000008015fc010: sub $0x48,%rsp
0x00000008015fc014: mov %rbx,0x40(%rsp)
0x00000008015fc019: mov %r14,0x38(%rsp)
0x00000008015fc01e: mov %r15,0x30(%rsp)
0x00000008015fc023: fnstcw 0xe(%rsp)
0x00000008015fc027: movb $0x2,0xf(%rsp)
0x00000008015fc02c: fldcw 0xe(%rsp)
0x00000008015fc030: movq $0x0,0x18(%rsp)
0x00000008015fc039: lea -8120048(%rip),%rax # 0x800e3d950
<_pthread_cond_wait+176>
0x00000008015fc040: mov %rax,0x10(%rsp)
0x00000008015fc045: movl $0x2,0x20(%rsp)
0x00000008015fc04d: lea 0x10(%rsp),%rbx
0x00000008015fc052: mov (%rbx),%rax
0x00000008015fc055: mov (%rax),%rax
0x00000008015fc058: mov %rbx,%rdi
0x00000008015fc05b: callq *%eax

The bug, I think, is at the lea instruction labeled as _pthread_cond_wait+176.
Looking at the assembler produced by llc, this is supposed to be putting the
address of _ZTV7counter into %rax. The offset relative to the PC is being
truncated to 32-bits, though not exactly oddly, and that appears to be the
problem. The corrupted vtable address causes the virtual method call to crash.
Absolute addressing must be used.

There might be a problem with the final instruction. In the llc assembler
output, the instruction is "callq *%rax", but this might be a bug in gdb's
dissassembler. Regardless, the contents of %rax at this point is clearly garbage.

It appears it fails on FreeBSD and not elsewhere because it reserves address
ranges differently.

[llvmbot]

A ulimit of under 2GB appears to work around the problem by forcing everything
to be allocated within 2GB of each other. This may or may not be a reason it
doesn't reproduce on other systems.

[llvmbot]

Jeff, please try something simple. Hello world, and simple programs that load /
store global variables, simple programs that call external functions, etc. If
something is fundamentally broken, it should be easy to come up with a 2-3 line
test case. I just need a bytecode file then.

[lattner]

we don't have a testcase to repro this. Please reopen when information to reproduce is available.