New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Using an OCaml library from multithreaded Python code #76
Comments
Sorry for the delay, and thank you very much for the question, I wasn't even aware of the existence of |
@jonathan-laurent maybe you can close this issue if it answers your question |
Sorry for the delay, I just noticed me being mentioned here. Most of our pyml use cases rely on multithreaded OCaml (because of using async and its thread pool). We run into issues when this interacts with Python threads, I've tried various things around calling |
@LaurentMazare This is interesting, thanks. To be clear though, do you think Async has something to do with the problems you observed or do you think they arise more generally? |
I think it's likely to be a problem with code that links the threads module (async and lwt jump to mind). Based on what we saw, even if the wrapped functions do not use async themselves, linking it was sufficient to result in some segfault when calling the functions from different Python threads. |
Thanks for developing and maintaining this great package!
I am currently using
pyml
andpythonlib
to call an OCaml library from Python. My Python application is using multiple threads (usingconcurrent.futures.ThreadPool
), although these threads never run in parallel (thanks to the GIL).So far, things have been working fine for me. However, I found this in the OCaml documentation:
https://ocaml.org/manual/intfc.html#ss:c-thread-register
Does pyml need to do anything special to deal with these? My application seems to be doing fine right now but I am worried about subtle bugs or leaks in the future.
@LaurentMazare Do you have any experience calling ocaml libraries from multithreaded python code at Janestreet?
The text was updated successfully, but these errors were encountered: