Reduce redundant module linking #3228
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3228 +/- ##
==========================================
+ Coverage 80.57% 80.59% +0.02%
==========================================
Files 390 390
Lines 78248 78349 +101
Branches 8834 8854 +20
==========================================
+ Hits 63045 63145 +100
+ Misses 13821 13808 -13
- Partials 1382 1396 +14 |
sklam
changed the title
There was a problem hiding this comment.
Thanks for this patch. The implementation makes sense and should improve performance especially for more involved compilations. As discussed over IRC, this should be threadsafe by virtue of the global compiler lock and atomic append/pop on list. Couple of minor comments but otherwise good to go. Thanks.
| return cres | ||
|
|
||
| def compile_subroutine(self, builder, impl, sig, locals={}): | ||
| def compile_subroutine(self, builder, impl, sig, locals={}, flags=None, | ||
| caching=True): |
There was a problem hiding this comment.
Perhaps add what caching does to the docstring for this function?
| def active_code_library(self): | ||
| """Get the active code library | ||
| """ | ||
| return self._codelib_stack[-1] |
There was a problem hiding this comment.
Could someone end up calling this before the _codelib_stack contains at least one entry? Worth catching it?
There was a problem hiding this comment.
If that happens, it is an internal error.
|
@sklam: This PR needs to resolve conflicts with master now. |
# Conflicts: # numba/compiler.py # numba/targets/base.py
|
Thanks for the fix ups, this can be merged as convenient. |
Reduce compilation time by removing redundant module linking. Avoid putting dependent module into global list of modules to link. Observed 5x reduction of
link_incalls for the "important" tests.Adds a stack of CodeLibrary into the target-context, which can now query the active code-library via
. active_code_library.Note about the "global list of modules to link":
The CodeGen maintains a list of modules required for all new compilation unit for that target. These modules are linked-into all future functions. Before this patch, any function created from
compile_internalis being added to the global list instead of the local one. This means any new function will have to link against anycompile_internalfunction regardless of whether they needed it or now.