FEA Support custom lib controllers #138
Conversation
jeremiedbb
changed the title
There was a problem hiding this comment.
A few comment on the doc, but the implem is super clean!!
I like this very much :)
Thanks @jeremiedbb
…reating an instance
There was a problem hiding this comment.
Once the doc is complete, and assuming the CI is fixed (ModuleNotFoundError: No module named 'mesonpy', apparently a new build time dependency for the BLIS CI entries) +1 for merge.
I like the new design, it's much cleaner in addition to making it possible to register a third party lib handler.
this error is because I tried to set |
|
Have you tried to mamba / pip install ninja? It might be a new runtime dependency of numpy when building in editable mode. |
jeremiedbb
changed the title
|
I added all the doc I wanted to add. This is ready for another round of reviews :) |
There was a problem hiding this comment.
A few more improvement. @jeremiedbb feel free to merge if you agree with them.
| identify the library. e.g. "libopenblas" for libopenblas.so. | ||
|
|
||
| and implement the following methods: `get_num_threads`, `set_num_threads` and | ||
| `get_version`. |
There was a problem hiding this comment.
| `get_version`. | |
| `get_version`. | |
| A library controller can optionally specify the `check_symbols` attribute valued | |
| with a tuple of `str` in cases where the `filename_prefixes` attribute is not | |
| specific enough to unambiguously identify a dynamically linked native library. |
There was a problem hiding this comment.
This is not True yet. We only check the symbols of libblas on windows to workaround conda renaming. We could try to implement support for that but I'd rather do that in a separate PR.
There was a problem hiding this comment.
Let's keep that for a follow-up PR then.
| the filename of each library with the `filename_prefixes`. If a match is found, a | ||
| controller is instantiated and a handler to the library is stored in the `dynlib` | ||
| attribute as a `ctypes.CDLL` object. It can be used to access the necessary symbols | ||
| of the shared library to implement the above methods. |
There was a problem hiding this comment.
| of the shared library to implement the above methods. | |
| of the shared library to implement the above methods assuming the optional | |
| preconditions defined in the `check_symbols` attribute are met. |
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
3.2.0 (2023-07-13) ================== - Dropped support for Python 3.6 and 3.7. - Added support for custom library controllers. Custom controllers must inherit from the `threadpoolctl.LibController` class and be registered to threadpoolctl using the `threadpoolctl.register` function. joblib/threadpoolctl#138 - A warning is raised on macOS when threadpoolctl finds both Intel OpenMP and LLVM OpenMP runtimes loaded simultaneously by the same Python program. See details and workarounds at https://github.com/joblib/threadpoolctl/blob/master/multiple_openmp.md. joblib/threadpoolctl#142
Fixes #137
The goal is to expose the
LibControllerabstract class, and custom controllers should inherit from it. This PR also exposes a new functionregisterthat simply adds the custom controller to the list of controllers to try.TODO: