Starting and stopping the runtime repeatedly seems to leak resources/memory on some systems

[msimberg]

On some systems (but not all) a test program like

#include <pika/init.hpp>

int main() {
    while (true) {
        pika::start(nullptr, 0, nullptr);
        pika::finalize();
        pika::stop();
    }
}

will eventually run out of memory (typically failing while trying to allocate thread stacks). Changing the pika.thread_queue.init_threads_count configuration option (easily changed with the environment variable PIKA_THREAD_QUEUE_INIT_THREADS_COUNT environment variable) changes the behaviour. The option controls how many thread stacks are pre-initialized at startup. With a high value the failure happens quickly and with a low value the failure is significantly delayed. This would indicate a leak of thread stacks. However, from the looks of it stacks are correctly released at shutdown. Additionally, on some systems memory does not seem to leak. This could indicate a difference in behaviour of mmap/munmap. When PIKA_WITH_THREAD_STACK_MMAP=OFF there seem to be no leaks supporting the mmap/munmap theory. What exactly could be causing this is not clear.