Deadlock when building in Release mode

[eschnett]

My HPX application is now working, using the fixing_588 branch, when I build in Debug mode. (This is independent of the OS thread issue described elsewhere.)

However, it hangs in Release mode. I start with

./bin/block_matrix --hpx:1:pu-offset=8 --hpx:threads=4 --hpx:pu-step=2 --hpx:print-bind --hpx:debug-hpx-log=block_matrix.log

and this hangs after some time, using ~600% CPU time, i.e. significantly more than the 400% I expect. A backtrace shows

#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x00007ffff610665c in boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service::thread_info&, boost::asio::detail::op_queue<boost::asio::detail::task_io_service_operation>&, boost::system::error_code const&) ()
   from /xfs1/eschnetter/compute/hpx/lib/hpx/libhpx.so.0
#2  0x00007ffff6106d0e in boost::asio::detail::task_io_service::run(boost::system::error_code&) () from /xfs1/eschnetter/compute/hpx/lib/hpx/libhpx.so.0
#3  0x00007ffff626726b in hpx::util::io_service_pool::thread_run(unsigned long)
    () from /xfs1/eschnetter/compute/hpx/lib/hpx/libhpx.so.0
#4  0x00007ffff5f01070 in hpx::runtime_impl<hpx::threads::policies::local_priority_queue_scheduler<boost::mutex>, hpx::threads::policies::callback_notifier>::wait() () from /xfs1/eschnetter/compute/hpx/lib/hpx/libhpx.so.0
#5  0x00007ffff5ef56f5 in hpx::runtime_impl<hpx::threads::policies::local_priority_queue_scheduler<boost::mutex>, hpx::threads::policies::callback_notifier>::run(hpx::util::function_nonser<int ()> const&) ()
   from /xfs1/eschnetter/compute/hpx/lib/hpx/libhpx.so.0
#6  0x00007ffff5e8fad1 in hpx::detail::run(hpx::runtime&, int (*)(boost::program_options::variables_map&), boost::program_options::variables_map&, hpx::runtime_mode, hpx::util::function_nonser<void ()> const&, hpx::util::function_nonser<void ()> const&) () from /xfs1/eschnetter/compute/hpx/lib/hpx/libhpx.so.0
#7  0x00007ffff5e900ad in hpx::detail::run_priority_local(hpx::util::function_nonser<void ()> const&, hpx::util::function_nonser<void ()> const&, hpx::util::command_line_handling&, bool) ()
   from /xfs1/eschnetter/compute/hpx/lib/hpx/libhpx.so.0
#8  0x00007ffff5e90b7c in hpx::run_or_start(int (*)(boost::program_options::variables_map&), boost::program_options::options_description const&, int, char**, std::vector<std::string, std::allocator<std::string> > const&, hpx::util::function_nonser<void ()> const&, hpx::util::function_nonser<void ()> const&, hpx::runtime_mode, bool) () from /xfs1/eschnetter/compute/hpx/lib/hpx/libhpx.so.0
#9  0x00007ffff5e91072 in hpx::init(int (*)(boost::program_options::variables_map&), boost::program_options::options_description const&, int, char**, std::vector<std::string, std::allocator<std::string> > const&, hpx::util::function_nonser<void ()> const&, hpx::util::function_nonser<void ()> const&, hpx::runtime_mode)
    () from /xfs1/eschnetter/compute/hpx/lib/hpx/libhpx.so.0
#10 0x0000555555adfe09 in hpx::init(int (*)(boost::program_options::variables_map&), boost::program_options::options_description const&, int, char**, hpx::util::function_nonser<void ()> const&, hpx::util::function_nonser<void ()> const&, hpx::runtime_mode) ()
#11 0x0000555555ae074a in main ()

[hkaiser]

This is the backtrace of only one thread, which doesn't tell anything. Nevertheless, I'll try to reproduce this issue here.

[eschnett]

I assume that this deadlock was connected to some of the other problems I was seeing.