D dynamic compilation support #2293

Hardcode84 · 2017-08-27T09:52:50Z

Here is current state of my 'jit' work. Simple examples work but everything mostly untested (and I'm not sure how properly test it). Tested on win64 and linux64. IR preparation for jit mostly in gen/runtimecompile.cpp. Jit itself implemented as shared library in runtime/rtcompile-rt/cpp-so/, it was mostly copypasted from llvm examples and I'm not sure it 100% correct.

Status:

Tests:

I am using existing lit infrastructure, not sure it is best suitable here
It is hard to check functions were actually dynamically compiled (if we compile them statically most of the tests still pass)
Need more tests

Platform support:

Win64:

Thread locals in dynamic code doesn't work

Relocation type not implemented yet!
UNREACHABLE executed at c:\ldc\llvm\lib\executionengine\runtimedyld\Targets/Runt
imeDyldCOFFX86_64.h:104!

Throwing and cathing exception in dynamic code doesnt work due to llvm bug (https://bugs.llvm.org//show_bug.cgi?id=24233)

Linux:

Thread locals in dynamic code doesn't work

LLVM ERROR: Cannot select: 0x1007ec8: i64 = X86ISD::WrapperRIP TargetGlobalTLSAddress:i64<i64* @_D12thread_local8threadIdm> 0 [TF=10]
  0x1007e60: i64 = TargetGlobalTLSAddress<i64* @_D12thread_local8threadIdm> 0 [TF=10]

MacOS:

Thread locals doesn't work

Areas of improvement:

Revise dynamic code optimization (copypasted from ldc/clang for now)
Research hotpatching for dynamic code dispatch (does llvm support this?)
Option to limit dynamic functions search depth
Extend @runtimeCompile attribute to allow mark some functions always non-dynamic
Get rid of thunks when calling dynamic functions from other dynamic functions
Check dynamic compilation with code coverage (theoretically it should work)
Check dynamic compilation with PGO (dynamic compiler can use PGO info collected offline)
Research into jitted partial application, eg:

@rumtimeCompile int foo(int a, int b)
{
  if (b == 1)
  {
    return a * 42;
  }
  return a + 42;
}


// Generate another function which interprets 'b' as constant allow llvm optimizer to drop branch completely
auto fun = apply!foo(placeholder, 0);
return fun(5); // foo(5, 0)

Throw exception when attempted to call dynamic functions without compilation

Hardcode84 · 2017-08-27T09:57:48Z

Related druntime PR: ldc-developers/druntime#101

thewilsonator · 2017-08-27T09:54:59Z

.gitmodules

@@ -1,6 +1,7 @@
 [submodule "druntime"]
 	path = runtime/druntime
-	url = https://github.com/ldc-developers/druntime.git
+	url = https://github.com/Hardcode84/druntime.git
+	branch = runtime_compile


Revert this, I'll pull the druntime PR as soon as you add a doc comment to it.

thewilsonator · 2017-08-27T10:09:59Z

ddmd/globals.h

@@ -234,6 +234,8 @@ struct Param
    uint32_t hashThreshold; // MD5 hash symbols larger than this threshold (0 = no hashing)

    bool outputSourceLocations; // if true, output line tables.
+
+    bool enableRuntimeCompile;


A small comment like the above would be nice.

…bled

Hardcode84 · 2017-08-29T22:17:49Z

So here some design overview of dynamic compilation feature

@runtimeCompie attribute can be applied to any function (including lambdas and class virtual methods)

Compiler part:

In DtoDeclareFunction read function uda's, if there is a @runtimeCompie attribute set IrFunction::runtimeCompile = true
If IrFunction::runtimeCompile == true, call for this function declareRuntimeCompiledFunction which will create thunk function rtCompileFunc
rtCompileFunc has same signature and attributes as the original function and any attemt to call or take address of the this function will be redirected to rtCompileFunc in DtoCallee
In DtoDefineFunction call defineRuntimeCompiledFunction for any function which have runtimeCompile == true
In defineRuntimeCompiledFunction create global variable which will store pointer to compiled function and create thunk function body which just calls this global var
(TODO: It is possible to get rid of this global var, we just need to hotpatch code in runtime with compiled code address, is it worth it? how to do this in llvm?)
In writeAndFreeLLModule call function generateBitcodeForRuntimeCompile for each module
In generateBitcodeForRuntimeCompile first recursively search functions calls inside functions body, starting with @runtimeCompie functions.
After that we have two lists of functions:

Directly marked by user @runtimeCompie
Not marked by user but used by @runtimeCompie functions directly or indirectly and with body available
Both lists are subject to dynamic compilation. For the first list dynamic version will be always used. For the second static version will be used when called from static code and dynamic version when called from dynamic code.
Also we now have a list of symbols required by dynamic functions (variables and external functions)
(TODO: Option for limit recursion depth?)

Clone module via llvm::CloneModule keeping only functions marked for dynamic compilations (directly or indirectly)
Optimize dynamic functions thunks in cloned module, make thunk global var constant and initialize it with function (fixupRtThunks) (TODO: llvm optimizer can handle this but better to remove these thunks completely in dynamic code)
Create module data required for dynamic compilation

/// Symbol required by dynamic code
struct RtCompileSymList
{
  const char* name; // Symbol name
  void* sym;        // Symbol address
};

/// Dynamic function
struct RtCompileFuncList
{
  const char* name; // Function name
  void* func;       // Thunk global var address to store compiled function pointer
};

/// Runtime compiled functions info per module
/// organized in linked list 
/// starting with `RuntimeCompileModulesHeadName` (defined in runtime)
struct RtComileModuleList
{
  RtComileModuleList* next; // Next module
  const void* irData;       // Ir data
  int32 irDataSize;         // Ir data size in bytes
  RtCompileFuncList* funcList; // Dynamic functions
  int32 funcListSize;            // Dynamic functions count
  RtCompileSymList* symList;   // Symbols
  int32 symListSize;             // Symbols count
};

Add module constructor which registers RtComileModuleList into global linked list

Runtime part:

Defines runtime compiled moduleds list head
Defines rtCompileProcessImplSo function which do actual compilation
rtCompileProcessImplSo for each module in list:
Parses ir data into llvm module
Optimizes module (TODO: optimization setup copypasted from clang and LDC itself and need to revise)
Add modules to jit (copypasted from llvm tutorials), resolve functions using RtComileModuleList data and update thunk vars

dnadlinger · 2017-08-29T23:11:19Z

Thanks a lot for this, this is impressive work! I hope I can find time for a closer review soon, but I'm sure the others will weigh in as well if you just ping them often enough. Do you keep a todo-list somewhere looking at actually shipping this on multiple OSes, etc.?

Hardcode84 · 2017-09-03T13:43:56Z

I tested this on linux and there is some issue with thread locals, except that everything seems to work.
I don't have device to test on macOS.

First comment updated.

JohanEngelen · 2017-09-04T20:47:20Z

ddmd/globals.h

@@ -234,6 +234,8 @@ struct Param
    uint32_t hashThreshold; // MD5 hash symbols larger than this threshold (0 = no hashing)

    bool outputSourceLocations; // if true, output line tables.
+
+    bool enableRuntimeCompile; // Enable dynamic compilation


this is not needed (we try to reduce the frontend diff (i.e. changes in ./ddmd/) as much as possible). Instead you should define cl::opt<bool> enableRuntimeCompile, such that it does not use "external" storage.

kinke · 2017-09-04T23:05:23Z

runtime/jit-rt/DefineBuildJitRT.cmake

@@ -0,0 +1,93 @@
+if(LDC_RUNTIME_COMPILE)
+    find_package(LLVM 3.9 REQUIRED support core irreader executionengine passes target nativecodegen)


I guess this conflicts with https://github.com/ldc-developers/ldc/blob/master/CMakeLists.txt#L22 and thus prevents successful linking.

kinke · 2017-09-04T23:12:54Z

runtime/CMakeLists.txt

@@ -600,11 +601,17 @@ macro(build_all_runtime_variants d_flags c_flags ld_flags path_suffix outlist_ta
        get_target_suffix("" "${path_suffix}" target_suffix)
        set_common_library_properties(ldc-profile-rt${target_suffix})
    endif()
+
+    build_jit_runtime ("${d_flags}" "${c_flags}" "${ld_flags}" "${path_suffix}" ${outlist_targets})


Any reason you're not including ${D_FLAGS} and maybe ${D_FLAGS_RELEASE} here?

kinke · 2017-09-05T19:40:13Z

runtime/jit-rt/DefineBuildJitRT.cmake

@@ -1,5 +1,5 @@
 if(LDC_RUNTIME_COMPILE)
-    find_package(LLVM 3.9 REQUIRED support core irreader executionengine passes target nativecodegen)
+    find_package(LLVM 3.7 REQUIRED support core irreader executionengine passes target nativecodegen)


What I meant was that invoking find_package() twice probably doesn't work and that conditionally including your additional libs in the single invokation should, although at the cost of less modularity.

I noticed another thing, it fail to find 32-bit llvm libraries when MULTILIB is ON, but as I understand compiler itself will still be 64 bit in this case. Does these machines have 32-bit llvm libs?

The compiler will be 64-bit, no 32-bit LLVM available and the 32-bit libs definitely shouldn't be needed; only the LDC executable (edit: and tools/utils? all 64-bit anyway) needs to be linked to LLVM. So this would indicate that some compiler/linker flags leak into other build targets (e.g., the 32-bit shared runtime libs).

I do dynamic compilation through llvm in my jit runtime shared library, which then linked to the target application, and if this application is 32-bit then jit runtime and llvm libs also have to be 32-bit.

Ah yes that makes perfect sense. I guess you'll have to build the native JIT runtime library exclusively then for now, excluding the 32-bit one for MULTILIB builds.

JohanEngelen · 2017-09-05T20:19:06Z

driver/linker-gcc.cpp

@@ -340,10 +340,12 @@ void ArgsBuilder::build(llvm::StringRef outputPath,
    args.push_back("-lldc-profile-rt");
  }

-  if (global.params.enableRuntimeCompile) {
+#if defined(LDC_RUNTIME_COMPILE)


nitpick: to prevent all these #if things, you could create a function isRuntimeCompileEnabled() similar to isUsingLTO().

Hardcode84 · 2017-11-09T17:46:25Z

druntime submodule update:
#2404

kinke · 2017-11-09T17:49:13Z

Why a separate PR? Just update the submodule in here.

Hardcode84 · 2017-11-09T17:51:15Z

ok

Hardcode84 · 2017-11-09T19:14:09Z

Renamed EVERYWHERE

kinke · 2017-11-10T00:17:37Z

Sorry to bother you hopefully one last time, but there's still docs/runtime_compile.md left to be renamed. ;)

dnadlinger · 2017-11-10T01:26:00Z

CMakeLists.txt

+#
+set(LDC_DYNAMIC_COMPILE False)  # must be a valid Python boolean constant (case sensitive)
+if (NOT (LDC_LLVM_VER LESS 400))
+    message(STATUS "Building LDC with runtime compilation support")


"with dynamic compilation support" for consistency.

dnadlinger · 2017-11-10T01:30:32Z

LICENSE

@@ -23,6 +23,9 @@ Libraries (lib/ and import/ in binary packages):
   Phobos (runtime/phobos), is distributed under the terms of the Boost
   Software License. See the individual source files for author information.

+ - The jit runtime library is distributed under the terms of the Boost


I'd make this "The ldc-jit-rt [or whatever library it really is] library is distributed under the terms of the Boost Software license (but additionally makes use of LLVM code)." or something like that for clarity – unless I'm missing something, people need to distribute the LLVM copyright notice when they are using dynamic compile support.

dnadlinger · 2017-11-10T01:31:33Z

runtime/jit-rt/cpp-so/optimizer.cpp

+    var->setConstant(true);
+    var->setInitializer(initializer);
+    var->setLinkage(llvm::GlobalValue::PrivateLinkage);
+    //    auto tempVar = new llvm::GlobalVariable(


What's the deal with this (and other) commented out sections?

dnadlinger · 2017-11-10T01:32:22Z

runtime/jit-rt/d/ldc/dynamic_compile.d

+/**
+ * Contains dynamic compilation API.
+ *
+ * Copyright: Authors 2017


This should probably at least say something vague like "the LDC team", or your name.

This was copypasted from somewhere, fixed to "the LDC team". But I don't like these 'authors' fields because everyone forget to update them.

Hardcode84 · 2017-11-10T19:14:32Z

More renamings and cleanups

kinke · 2017-11-10T19:30:14Z

Thanks Ivan. I'll merge this tomorrow ~~afternoon~~ evening (CET) before tagging v1.6.0-beta1 if there are no objections.

dnadlinger · 2017-11-11T20:07:05Z

Ouch, I would have very much preferred this being merged in less than 93 commits.

kinke · 2017-11-11T20:24:03Z

Feasible options being 1 or 93, I opted for the full bunch. I'll gladly let others make decisions like this in the future, I gave a 24+h notice.

dnadlinger · 2017-11-11T20:27:31Z

It's not a huge thing either way, but there is also no need to react in a snarky fashion.

Hardcode84 · 2017-11-11T20:47:09Z

Thanks for review. Actually, I wanted to ask about squashing but I forgot.

thewilsonator reviewed Aug 27, 2017

View reviewed changes

Hardcode84 added 5 commits August 27, 2017 17:42

D dynamic compilation support

42f283c

Some unused code cleanup

0ef4edb

Export LDC_RuntimeCompilation version when runtime compilation is ena…

482b822

…bled

Revert submodule

6b8fee3

llvm 5.0 support

a893405

Hardcode84 force-pushed the runtime_compile_v5 branch from bb143c2 to a893405 Compare August 27, 2017 15:16

Comment

0c7ec10

Hardcode84 added 5 commits September 3, 2017 13:40

more comments

42adb03

jit runtime cmake fix

e82aeb6

more cleanup and cmake fixes

cc1e8ba

Check personality function in enumFuncSymbols

087a9a8

Merge remote-tracking branch 'origin/master' into runtime_compile_v5

81532cd

Hardcode84 added 3 commits September 3, 2017 16:44

Fix jit cmake

1063fde

more tests

6aa5a8e

rename rtCompileProcess() to compileDynamicCode()

5bb48d8

JohanEngelen reviewed Sep 4, 2017

View reviewed changes

kinke reviewed Sep 4, 2017

View reviewed changes

Hardcode84 added 4 commits September 5, 2017 21:23

Do not use exteranl storage for enableRuntimeCompile flag

a9f6875

pass ${D_FLAGS};${D_FLAGS_RELEASE} to build_jit_runtime

1b8e94c

attempt to fix cmake

63ffec8

Merge remote-tracking branch 'origin/master' into runtime_compile_v5

daee5c6

kinke reviewed Sep 5, 2017

View reviewed changes

WIP: another attempt to cmake, please ignore

658940c

JohanEngelen reviewed Sep 5, 2017

View reviewed changes

Hardcode84 added 12 commits November 9, 2017 20:56

remove unneeded .setRelocationModel(llvm::Reloc::Static)

a6c88bf

Rename attributes

eb370e1

rename command line option

8a6a549

Rename module

cd5ba38

Renaming

33c6cbb

Renaming

876d37f

Renaming

e53f7f4

rename compiler files

5426ffb

Rename functions

c839e9d

Rename

1c1b645

clang-format

911cf5b

Renaming

adca0a4

dnadlinger reviewed Nov 10, 2017

View reviewed changes

Hardcode84 added 2 commits November 10, 2017 21:36

remove commented code

347c30d

Renamings

b374568

kinke merged commit e8405a1 into ldc-developers:master Nov 11, 2017

		@@ -0,0 +1,93 @@
		if(LDC_RUNTIME_COMPILE)
		find_package(LLVM 3.9 REQUIRED support core irreader executionengine passes target nativecodegen)

D dynamic compilation support #2293

D dynamic compilation support #2293

Conversation

Hardcode84 commented Aug 27, 2017 • edited Loading

Hardcode84 commented Aug 27, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Hardcode84 commented Aug 29, 2017 • edited Loading

dnadlinger commented Aug 29, 2017

Hardcode84 commented Sep 3, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kinke Sep 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Hardcode84 commented Nov 9, 2017

kinke commented Nov 9, 2017

Hardcode84 commented Nov 9, 2017

Hardcode84 commented Nov 9, 2017

kinke commented Nov 10, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Hardcode84 commented Nov 10, 2017

kinke commented Nov 10, 2017 • edited Loading

dnadlinger commented Nov 11, 2017

kinke commented Nov 11, 2017

dnadlinger commented Nov 11, 2017

Hardcode84 commented Nov 11, 2017

Hardcode84 commented Aug 27, 2017 •

edited

Loading

Hardcode84 commented Aug 29, 2017 •

edited

Loading

kinke Sep 5, 2017 •

edited

Loading

kinke commented Nov 10, 2017 •

edited

Loading

kinke commented Nov 10, 2017 •

edited

Loading