Add ORC-MCJIT replacement binding

[sklam]

Adds binding to OrcMCJITReplacement.

Progress:

• write binding
• write tests
• write docs

Note:

The Orc-MCJIT is not a direct replacement for MCJIT for all cases. When testing in numba, the lazy compilation in Orc-MCJIT seems to be non-threadsafe and it is failing any parallel execution of the same function. (See related LLVM issue https://bugs.llvm.org//show_bug.cgi?id=5184). In Orc-MCJIT, it seems the lazy compilation cannot be turned off.

[codecov-io]

Codecov Report

Merging #245 into master will increase coverage by 0.03%.
The diff coverage is 95.65%.

@@            Coverage Diff             @@
##           master     #245      +/-   ##
==========================================
+ Coverage   91.87%   91.91%   +0.03%     
==========================================
  Files          33       33              
  Lines        4728     4774      +46     
  Branches      330      332       +2     
==========================================
+ Hits         4344     4388      +44     
- Misses        308      309       +1     
- Partials       76       77       +1

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8047388...9bcf87c. Read the comment docs.

[coveralls]

Coverage increased (+0.04%) to 93.527% when pulling 9bcf87c on sklam:feature/orcjit into 8047388 on numba:master.

[seibert]

Does this mean we'll need to write our own MCJIT replacement with the ORC API that is actually thread-safe?

[sklam]

Yes. I think there are two potential approaches:

1. Make a eager-version. The problem comes from ORC-MCJIT being lazy. It puts a stub function that defers the compilation to 1st call. The stub is not threadsafe though. Making it eager should remove the problem.

2. Make a threadsafe compile-on-demand layer ourselves.

[seibert]

Do we benefit from the lazy compilation, since we are already deferring our call to LLVM to the first function call anyway? If no benefit, then option 1 seems to make the most sense.

[seibert]

And I assume that option 1 means Numba (or llvmlite?) will still need to enforce locking at the Python level to prevent concurrent compilation.

[sklam]

The current numba setup won't benefit from lazy compilation. I prefer option 1 and hope that it actually fixes the issue. Option 1 will be exactly like what MCJIT is doing---eager compilation all the time. Numba already has the concurrent compilation lock.

[pitrou]

Wow. Is that a documented difference in LLVM? A bug?

[sklam]

Is the behavior in the MCJIT-version documented?

It seems OrcJIT is stricter here. Due to its laziness, Orc first tries to find a IR module that has the symbol. Once found, it checks if it is already compiled and do its thing. If not found, it stops and return NULL.

[alendit]

What's the status of this? AFAIK ORC supports eager compilation via the basic IRCompileLayer. Maybe the OrcMCJITReplacement isn't the best way forward? I could add ORC as a separate ExecutionEngine with a superset of MCJIT's python API. This way the numba wouldn't be directly affected and the new EE could be thoroughly tested before any switch is made.

[lhames]

@abebeos OrcMCJITReplacement was intended as an OrcV1 drop-in replacement for MCJIT. has been deprecated for a while and was removed with the rest of OrcV1 in LLVM 12. I think this PR can be closed.

[esc]

sure