Is dlib Python API releasing GIL when calls C++ functions?

[meownoid]

I'm wondering if I can use dlib with python multithreading.

[supervacuus]

The default behavior in pybind11 is to hold the GIL when calling C++ from Python:

http://pybind11.readthedocs.io/en/stable/advanced/misc.html#global-interpreter-lock-gil

There is currently no GIL-release/acquire in the python wrapper.

[davisking]

We could add statements to unlock the GIL, however, it's not clear to me what a python user then does with that. Any python code you write is going to reengage the GIL. So it doesn't seem like you can do very much. Or is there some important use case that would be enabled by this?

[meownoid]

Consider a web service for face recognition. Each client gets it's own thread, something like this:

@run_async
def generate_response(request):
    photo = load_photo(request)  # IO, releases GIL
    photo = process_photo_with_pillow(photo)  # releases GIL
    person = recognize_face_with_dlib(photo)  # oops
    upload_result(photo, person)  # IO, releases GIL

So, if generate_response is running in separate thread using threading module, GIL is acquired just for little amount of time between the IO and C-calls. Except for the dlib which enforces all other user's threads to wait. This is a real example of project on that I'm currently working on and all other libraries (numpy, Pillow, requests) releases GIL, but not dlib. And I have to move all dlib calls to another process and think about communication and synchronization between processes...

[davisking]

Isn't that dlib call going to be CPU bound anyway? How are you going to have multiple threads running it? They are just going to compete for CPU resources. You probably won't get much higher throughput. But in any event, I'm not adverse to someone submitting PR that adds GIL release and acquire calls to these things. It should be pretty easy to do.

[meownoid]

How are you going to have multiple threads running it?

When GIL is released and dlib computation is running in several threads, they all running simultaneously on several cores.

When GIL is acquired only one of this threads can be running at this time and all others are waiting for it.

thread_1 = Thread(target=dlib_computation)
thread_2 = Thread(target=dlib_computation)

thread_1.start()
thread_2.start()

GIL acquired:

<=== thread 1 ===>                    <some Python code> <=== thread 1 ===>
                   <=== thread 2 ===>                                       <=== thread 2 ===>

GIL released:

<=== thread 1 ===> <some Python code> <=== thread 1 ===>
<=== thread 2 ===>                    <=== thread 2 ===>

So, I'm talking not about paralleling one dlib computation, but about running several dlib computations in parallel, which requires either threading + GIL releasing or multiprocessing.

[davisking]

Yes, I understand. But if the underlying thing in dlib you are calling is already using all the CPU cores, which it should be, then what you are talking about isn't going to do you any good. That's my point.

But in any case, if someone wants to implement this and submit a PR that's fine with me.

[meownoid]

But if the underlying thing in dlib you are calling is already using all the CPU cores

Does it? Sorry, I couldn't find any information about this in documentation. In my observations it uses only one core. Is there any parameters for internal multi-threading in Python API?

[davisking]

You should be linking to a multi-threaded BLAS library. Then it will use all the cores.

[dlib-issue-bot]

Warning: this issue has been inactive for 57 days and will be automatically closed on 2018-09-07 if there is no further activity.

If you are waiting for a response but haven't received one it's likely your question is somehow inappropriate. E.g. you didn't follow the issue submission instructions, or your question is easily answerable by reading the FAQ, dlib's documentation, or a Google search.

[dlib-issue-bot]

Notice: this issue has been closed because it has been inactive for 60 days. You may reopen this issue if it has been closed in error.

[zhangchn]

A single line of

 py::gil_scoped_release release;

at the beginning of each code block may help.

At least with this it keeps the main thread going and updating video captures in time.

[davisking]

Yes, that would do it. Someone who is using this and wants the feature should do this, test it and make sure it really hits their use case, and submit a PR.