Add an OpenFHE dialect interpreter#2314
Conversation
|
CC @eymay I'd be curious to see if this is suitable for the large examples you were working with. I suspect the resulting program latency will suffer slightly by interpreting it instead of compiling it, and I'd be curious to hear how much worse it is for your examples. |
|
One thing that can be improved here is to identify that an SSA value has had its last use executed, and remove it from the environment to free up memory. Otherwise we're just keeping all SSA values in RAM for the entirety of the program. |
This PR adds the ability to run OpenFHE dialect code in an interpreted mode, rather than generating C++ code that is then compiled by clang. This is intended to support cases in the codebase where clang compile times are very long due to having fully unrolled FHE kernels. We have unrolled FHE kernels because noise analysis and ciphertext management passes do not support loops yet, and the plan to add support for loops is based on the HALO compiler paper (see google#289). To use the interpreter, load the `openfhe_interpreter_test` bazel macro, and use the generated mlir-opt filename from that macro in your test file to load the openfhe dialect IR as a ModuleOp, which is passed to an Interpreter object. Then a call to ``` interpreter.interpret("<func_name>", <func_args>) ``` must have as its second argument a vector of `TypedCppValue` objects whose values correspond to the same ordered arguments of the function being interpreted. The call to `interpret` returns a vector of results as `TypedCppValue`. These can be chained together as in the compiled case to produce a full execution of the encrypt/run/decrypt flow.
Hi, I tried the IRs I had but because they are from an older version of HEIR it wasn't directly compatible and backporting the interpreter turned out to be more work than I imagined. So I ended up copying the matvec op from the example multiple times to get a stress test and instead compared its compiled and interpreted executions. Surprisingly I ended up with 1.5x slowdown in interpreted mode with a benchmark that has 20 matvecs. The main bottleneck is not the OpenFHE calls but rather the cleartext computations such as The solution to it would probably involve compiling most of the cleartext computations that do not involve FHE operations and let the interpreter do the FHE operations. JIT compiling the cleartext can be an option. Instead of interpreting cleartext lines, they can be translated and compiled just-in-time to functions that are run by the interpreter. Then it is also possible to cache these functions in case there are repetitions of these cleartext lines which should make the compilation overhead a one-time cost for the repeated cases. |
This PR adds the ability to run OpenFHE dialect code in an interpreted mode, rather than generating C++ code that is then compiled by clang.
This is intended to support cases in the codebase where clang compile times are very long due to having fully unrolled FHE kernels. We have unrolled FHE kernels because noise analysis and ciphertext management passes do not support loops yet, and the plan to add support for loops is based on the HALO compiler paper (see #289).
To use the interpreter, load the
openfhe_interpreter_testbazel macro, and use the generated mlir-opt filename from that macro in your test file to load the openfhe dialect IR as a ModuleOp, which is passed to an Interpreter object.Then a call to
must have as its second argument a vector of
TypedCppValueobjects whose values correspond to the same ordered arguments of the function being interpreted. The call tointerpretreturns a vector of results asTypedCppValue. These can be chained together as in the compiled case to produce a full execution of the encrypt/run/decrypt flow.Note: This PR was written with a decent amount of assistance from an LLM. Though it has unit tests and runs the Halevi-Shoup matvec test case correctly, it was not tested on BGV/BFV (just the one CKKS example) and so it should be expected to have some rough edges.