Windows DLL hangs on unload after using a server_iostream start_async()

[markcellis]

Before I take the time to create a full test case and bug report I just want to know if this is a known problem or if I'm doing something wrong.

The application I'm developing is a plugin to a flight simulator. The plugin is a dll that gets dynamically loaded by the simulator when it starts and then unloaded when the simulator exists or you request an unload and reload of all plugins.

The only part of dlib I'm using is an server_iostream. I start the server with start_async(); and then when I need to clean up before my plugin is unloaded I do a clear() on the server and then delete the server.

I think what is going on is there is a few stray dlib threads are still running or possibly an issue similar to what was reported in issue 505 where windows is killing threads on dll unload which then cause dlib static destructors to hang.

I'm using the dlib 19.24

My questions are:

1. Is there more I need to do to cleanly shut down a server_iostream?
2. Is there something I should be doing to properly clean up dlib static objects before a dll is unloaded?
3. Is this a known issue?

[davisking]

Yeah the way DLLs handle globals and threads can be frustrating. There is also this kind of thing https://github.com/davisking/dlib/blob/master/dlib/threads/threads_kernel_shared.cpp#L79. You could try making it actually wait on those threads.

I don't know of a good general way to deal with this in windows though.

[markcellis]

So I did some snooping in the Visual Studio debugger and I found that the destruct_if_ready() method is never called in the threader_destruct_helper when the dll is unloaded by the simulator but it is called when the entire simulator process ends.

    struct threader_destruct_helper
    {
        // cause the thread pool to begin its destruction process when 
        // global objects start to be destroyed
        ~threader_destruct_helper()
        {
            **thread_pool().destruct_if_ready();**
        }
    };

My theory is that windows must handle cleaning up static structs differently when a dll is unloaded vs when the encapsulating process ends.

    threader& thread_pool (
    ) 
    {
        static threader* thread_pool = new threader;
        **static threader_destruct_helper a;**
        return *thread_pool;
    }

I'm not sure if this causing my headaches or not.

This thread seems to outline the problem:

https://stackoverflow.com/questions/22500229/c-static-objects-in-dlls-not-having-destructors-called

I wonder if dlib should use something like a dlib::init() and dlib::release() where singletons are created in the init() function and then cleanly destroyed in the release() function.

Any thoughts?

[davisking]

I'm pretty violently opposed to needing to call global functions to setup and tear-down state :)

They make it pretty impossible to layer software on to of libraries like that without eventually running into conflicts about which subsystem is responsible for doing that setup and tear-down. And in most situations it's totally unnecessary, it's just with windows and DLLs there is this awful thing that was done a long time ago and now we have to live with it.

Anyway, take a look at http://dlib.net/compile.html#DLIB_THREAD_POOL_TIMEOUT. You can just set that to 0 and it will disable the thread pool. Then threads won't lurk in the background after they end.

[markcellis]

Ok, so I tried the THREAD_POOL_TIMEOUT set to 0 and it still has a problem unloading the DLL.

I spent some time debugging and after doing a break all I found one thread still running in my DLL. It is in the timer_global_clock thread and it is waiting for more work to do. See attached

As best I can tell the global clock is not getting deleted properly when a DLL is unloaded

I can see there was quite some discussion about this in issue 505:

#505

But it is hard to see if it ever got resolved. It looks like you closed the issue back in April of 2017 but I'm not sure I get what the solution was. At the top of timer.cpp there is a mention of implementing a global_delete_clock function. It sounded like sigbjorn tried to do that but it was not clear he was successful. His fork of dlib is no longer available to I can't tell.

[davisking]

Yeah there wasn't ever a good resolution. Does this make it work for you?

--- a/dlib/timer/timer.cpp
+++ b/dlib/timer/timer.cpp
@@ -49,11 +49,11 @@ namespace dlib
         // linux: the destructor for linux will do it's usual job regardless.
         //
 
-        #ifndef _WIN32
         m.lock();
         shutdown = true;
         s.signal();
         m.unlock();
+        #ifndef _WIN32
         wait();
         #endif
     }

I'm guessing no since the destructor there probably isn't run. You could add a global method that does that though to tell the thread to terminate. Or on windows that global clock could instead be a std::weak_ptr so that it gets destructed immediately when there aren't any client uses of it rather than waiting for a global destructor to do it. That's probably the best solution here. It would case there to be extra startup costs for timer objects if the whole program has just one and creates and destroys it over and over, but I doubt that is a common problem.