Prepare ORTools for static compilation in Rust #428
Conversation
PR Review ChecklistDo not edit the content of this comment. The PR reviewer should simply update this comment by ticking each review item below, as they get completed. Trivial Change
Code
Architecture
|
| @@ -1,3 +1,9 @@ | |||
| common --enable_platform_specific_config | |||
There was a problem hiding this comment.
We set different build flags per-host.
This is because OR-tools compilation requires C++20, and the flag to set this is different for windows MSVC compilers and non-MSVC compilers.
Note that this flag is another barrier to cross-compilation, if that ever comes back onto the table.
There was a problem hiding this comment.
This command basically is a note to bazel that it should should select platform-specific configurations according to the host platform
| load("//library/ortools/cc:deps.bzl", "google_or_tools") | ||
| google_or_tools() | ||
| load("@com_google_protobuf//:protobuf_deps.bzl", google_or_tools_protobuf_deps = "protobuf_deps") | ||
| google_or_tools_protobuf_deps() |
There was a problem hiding this comment.
or tools has its own protobuf dependencies
| visibility = ["//visibility:public"] | ||
| ) | ||
| """.format(lib) | ||
| def google_or_tools(): |
There was a problem hiding this comment.
or_tools and all of its dependencies. We get this from the template OR-tools bazel project: https://github.com/or-tools/bazel_or-tools
| git_repository( | ||
| name = "com_google_ortools", | ||
| remote = "https://github.com/google/or-tools.git", | ||
| tag = "v9.3", |
There was a problem hiding this comment.
we use a tag instead of commit + shallow_since to reduce the size of the initial git clone
| std::unique_ptr<MPSolver> new_mpsolver_sat() { | ||
| return std::unique_ptr<MPSolver>(MPSolver::CreateSolver("SAT")); |
There was a problem hiding this comment.
Expose the SAT interface explicitly
|
Moving back to draft until I can come back to this to finish it up |
What is the goal of this PR?
We modify the OR-Tools dependency to be statically compiled into a project, rather than linked dynamically. This leverages the fact that OR-Tools is a Google repository that exposes Bazel targets to build with. As a result, we follow the model we use for RocksDB, which is to statically compile a platform-native library into our project on demand.
Note that the OR-Tools libraries require C++20 to compile, which we set in a platform-specific way in the bazel configuration file (
.bazelrc)What are the changes implemented in this PR?
Justification
Like RocksDB, ORTools was initially considered as a dynamically linked library. However, we have discovered that the Rust+Bazel ecosystem does not support packaging dynamic libraries and binaries together elegantly, and is instead geared toward static compilation.
For RocksDB, we rely on the
rocksdb-syscrate to perform the compilation of RocksDB's C++ core into a Rust library. There is no such equivalent crate for ORTools, the existing crates are wrappers around dynamically linked ORTools distributions.The solution for now is to leverage our existing work to build our own CXX bridge between Rust and C++. Instead of building this bridge on top of header files and dynamic libraries that come pre-compiled, we utilise ORTools' Bazel targets to statically compile the library for the current platform.
Note: as a consequence of this, projects using OR-tools from this repository will be much slower: compiling OR-tools can take some time, on top of the time required to clone its Github repository (~200mb). However, the compile time can be cut drastically by using a bazel build cache.