global variables declared in module A but defined in module B are not resolved by the JIT

[llvmbot]

Bugzilla Link	2606
Version	trunk
OS	Linux
Attachments	modified HowToUseJIT.cpp which reproduces the problem
Reporter	LLVM Bugzilla Contributor
CC	@asl,@nlewycky

Extended Description

If you declare and use a global variable in module A and define it in module B the execution engine will not resolve the symbol in module A.

The attached modified version of HowToUseJIT.cpp shows this problem. You can manually work around this limitation by explicitly calling addGlobalMapping on the declaration using the same pointer that's mapped to the definition.

[lattner]

Hi Paul, can you propose a patch? This should be easy if you are interested.

[nlewycky]

*** Bug llvm/llvm-bugzilla-archive#3143 has been marked as a duplicate of this bug. ***

[nlewycky]

The duped bug, pr3143 has a proposed patch.

[nlewycky]

Benjamin Scarlet reports that my patch doesn't work for him due to recursive compilation.

I think that there's a second bug here; calling getPointerToFunction on F() that contains a call to F2() should be getting a stub for F2, not trying to compile the whole function.

[llvmbot]

2.7 version of testcase
This is a 2.7 version of HowToUseJIT.cpp which reproduces the problem.

[llvmbot]

test case which shows cross module function linkage problem
This is a test case which shows the issue when trying to find functions defined in other modules, without using ExecutionEngine::addGlobalMapping(...). This test case was only tested on OS 10.6.2 and Linux, gcc 4.1.2.

To build on OS X 10.6.2 use:

g++ -g twoModuleCallTest.cpp llvm-config --cppflags --ldflags --libs core \ jit native -o twoModuleCallTest

On Linux use:

g++ -g -rdynamic twoModuleCallTest.cpp llvm-config --cppflags --ldflags --libs core \ jit native -o twoModuleCallTest

[llvmbot]

a work in progress proposed patch
This patch is my first attempt at solving the above test cases. Although it works on 32bit LINUX (gcc 4.1.2), and 64bit OS X 10.6.2, the patch needs to be refined as it has taken some liberties with the code. I will also submit this to llvm.dev to start a discussion.

[llvmbot]

Replace use of JITResolver::forceEmitFunctionStub(...) with JITResolver::getLazyFunctionStub(...).
Modified 4240 with replacement of JITResolver::forceEmitFunctionStub(...) with JITResolver::getLazyFunctionStub(...) as per discussion in llvm.dev list. Olivier Meurant
pointed out flaws with attachment 4240's approach concerning functions. Other issues
still remain.

[llvmbot]

Forgot to revert JITResolver::JITCompilerFn(...) in 4254
Same comment as given in 4254 additionally with JITResolver::JITCompilerFn(...) reverted.

[llvmbot]

Created an attachment (id=4257) [details]
Forgot to revert JITResolver::JITCompilerFn(...) in 4254

Same comment as given in 4254 additionally with JITResolver::JITCompilerFn(...)
reverted.

Now that multiple execution engine instances can exist, a usable workaround involves having one execution engine per module, with LazyFunctionCreators installed to resolve cross-module calls and JIT them. I don't know how heavy each ExecutionEngine instance is, though.

Perhaps a better idea would be to add a LazyFunctionCreator variant that returns Function instances rather than native function pointers. Obviously that won't be helpful for native library lookup, but you can not only resolve IR functions cross module, but you can also emit IR functions on demand more easily.

[nlewycky]

mentioned in issue llvm/llvm-bugzilla-archive#3143