Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sheneos on_completed_bulk seg fault in distributed #490

Closed
brycelelbach opened this issue Aug 11, 2012 · 10 comments

Comments

Projects
None yet
2 participants
@brycelelbach
Copy link
Member

commented Aug 11, 2012

This happens when running in distributed on Linux, using the top of trunk::

Received Segmentation fault, 53 frames:
0x7fa150fd80d2  : boost::backtrace::backtrace(unsigned long) + 0x80 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa150fd8236  : boost::trace() + 0x1e in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa150fcc663  : hpx::detail::backtrace() + 0x18 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa15109ee47  : hpx::termination_handler(int) + 0x26 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa14b425030  : ??? + 0x7fa14b425030 in /lib/x86_64-linux-gnu/libpthread.so.0
0xc52f5c        : std::vector<double, std::allocator<double> >::capacity() const + 0xc in /home/wash/hpx/gcc-4.6.2-debug/bin/sheneos_test
0xc4f7af        : std::vector<double, std::allocator<double> >::operator=(std::vector<double, std::allocator<double> > const&) + 0x41 in /home/wash/hpx/gcc-4.6.2-debug/bin/sheneos_test
0x7fa153820d7e  : sheneos::on_completed_bulk::operator()(hpx::lcos::future<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >) + 0x15a in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpx_component_sheneosd.so.1
0x7fa1538537d1  : hpx::util::detail::vtable<false>::type<sheneos::on_completed_bulk, void (hpx::lcos::future<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >), void, void>::invoke_(void**, hpx::lcos::future<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, mpl_::bool_<false>) + 0x3a in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpx_component_sheneosd.so.1
0x7fa15384cea7  : hpx::util::detail::vtable<false>::type<sheneos::on_completed_bulk, void (hpx::lcos::future<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >), void, void>::invoke(void**, hpx::lcos::future<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >) + 0x3a in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpx_component_sheneosd.so.1
0x7fa15384d752  : hpx::util::function_base<void (hpx::lcos::future<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >), void, void>::operator()(hpx::lcos::future<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >) + 0x72 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpx_component_sheneosd.so.1
0x7fa1538779e1  : void hpx::lcos::detail::future_data<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_data<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&) + 0x189 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpx_component_sheneosd.so.1
0x7fa153872eeb  : hpx::lcos::detail::promise<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_value(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&) + 0x2f in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpx_component_sheneosd.so.1
0xbcb336        : hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_value_nonvirt(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&) + 0x3a in /home/wash/hpx/gcc-4.6.2-debug/bin/sheneos_test
0xcba0df        : hpx::threads::thread_state_enum hpx::actions::base_action1<hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, 1, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&, &hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_value_nonvirt, hpx::actions::direct_action1<hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, 1, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&, &hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_value_nonvirt, hpx::actions::detail::this_type>, (hpx::threads::thread_priority)0>::thread_function::operator()<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&>(unsigned long, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&) const + 0x155 in /home/wash/hpx/gcc-4.6.2-debug/bin/sheneos_test
0xcb477b        : hpx::threads::thread_state_enum hpx::util::detail::bound_functor2<hpx::actions::base_action1<hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, 1, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&, &hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_value_nonvirt, hpx::actions::direct_action1<hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, 1, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&, &hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_value_nonvirt, hpx::actions::detail::this_type>, (hpx::threads::thread_priority)0>::thread_function, unsigned long, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::operator()<hpx::threads::thread_state_ex_enum&>(hpx::threads::thread_state_ex_enum&) + 0x8b in /home/wash/hpx/gcc-4.6.2-debug/bin/sheneos_test
0xcb1456        : hpx::util::detail::vtable<false>::type<hpx::util::detail::bound_functor2<hpx::actions::base_action1<hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, 1, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&, &hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_value_nonvirt, hpx::actions::direct_action1<hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, 1, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&, &hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_value_nonvirt, hpx::actions::detail::this_type>, (hpx::threads::thread_priority)0>::thread_function, unsigned long, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), void, void>::invoke_(void**, hpx::threads::thread_state_ex_enum, mpl_::bool_<false>) + 0x25 in /home/wash/hpx/gcc-4.6.2-debug/bin/sheneos_test
0xcadf39        : hpx::util::detail::vtable<false>::type<hpx::util::detail::bound_functor2<hpx::actions::base_action1<hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, 1, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&, &hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_value_nonvirt, hpx::actions::direct_action1<hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, 1, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >&&, &hpx::lcos::base_lco_with_value<std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > >, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >::set_value_nonvirt, hpx::actions::detail::this_type>, (hpx::threads::thread_priority)0>::thread_function, unsigned long, std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > >, hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), void, void>::invoke(void**, hpx::threads::thread_state_ex_enum) + 0x23 in /home/wash/hpx/gcc-4.6.2-debug/bin/sheneos_test
0x7fa15114c86b  : hpx::util::function_base<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), void, void>::operator()(hpx::threads::thread_state_ex_enum) + 0x5b in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1511495e1  : hpx::actions::action_manager::fetch_parcel(hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&) + 0x5c1 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510d255b  : boost::_mfi::mf2<void, hpx::actions::action_manager, hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&>::operator()(hpx::actions::action_manager*, hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&) const + 0x6b in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510cf894  : void boost::_bi::list3<boost::_bi::value<hpx::actions::action_manager*>, boost::arg<1>, boost::arg<2> >::operator()<boost::_mfi::mf2<void, hpx::actions::action_manager, hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&>, boost::_bi::list2<hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&> >(boost::_bi::type<void>, boost::_mfi::mf2<void, hpx::actions::action_manager, hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&>&, boost::_bi::list2<hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&>&, int) + 0x86 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510cc816  : void boost::_bi::bind_t<void, boost::_mfi::mf2<void, hpx::actions::action_manager, hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&>, boost::_bi::list3<boost::_bi::value<hpx::actions::action_manager*>, boost::arg<1>, boost::arg<2> > >::operator()<hpx::parcelset::parcelhandler, hpx::naming::gid_type>(hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&) + 0x4c in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510c9381  : hpx::util::detail::vtable<false>::type<boost::_bi::bind_t<void, boost::_mfi::mf2<void, hpx::actions::action_manager, hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&>, boost::_bi::list3<boost::_bi::value<hpx::actions::action_manager*>, boost::arg<1>, boost::arg<2> > >, void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&), void, void>::invoke_(void**, hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&, mpl_::bool_<false>) + 0x2e in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510c4d11  : hpx::util::detail::vtable<false>::type<boost::_bi::bind_t<void, boost::_mfi::mf2<void, hpx::actions::action_manager, hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&>, boost::_bi::list3<boost::_bi::value<hpx::actions::action_manager*>, boost::arg<1>, boost::arg<2> > >, void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&), void, void>::invoke(void**, hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&) + 0x2f in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510d101d  : hpx::util::function_base<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&), void, void>::operator()(hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&) + 0x63 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510cd9aa  : boost::detail::function::void_function_obj_invoker2<hpx::util::function_nonser<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type const&)>, void, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>::invoke(boost::detail::function::function_buffer&, hpx::parcelset::parcelhandler&, hpx::naming::gid_type) + 0x3a in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510d6778  : boost::function2<void, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>::operator()(hpx::parcelset::parcelhandler&, hpx::naming::gid_type) const + 0x7e in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510d5dfc  : boost::signals2::detail::void_type boost::signals2::detail::call_with_tuple_args<boost::signals2::detail::void_type>::m_invoke<boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)>, 0u, 1u, hpx::parcelset::parcelhandler&, hpx::naming::gid_type&>(void*, boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)>&, boost::signals2::detail::unsigned_meta_array<0u, 1u>, std::tuple<hpx::parcelset::parcelhandler&, hpx::naming::gid_type&>) const + 0x4c in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510d4be0  : boost::signals2::detpbsdsh: task 2 exit status 262
ail::void_type boost::signals2::detail::call_with_tuple_args<boost::signals2::detail::void_type>::operator()<boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)>, hpx::parcelset::parcelhandler&, hpx::naming::gid_type&>(boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)>&, std::tuple<hpx::parcelset::parcelhandler&, hpx::naming::gid_type&>) const + 0x4c in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510d321a  : boost::signals2::detail::void_type boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>::m_invoke<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >(boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > const&, boost::signals2::detail::void_type const*) const + 0x56 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510d0f82  : boost::signals2::detail::void_type boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>::operator()<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >(boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > const&) const + 0x30 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510cd865  : boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> >::dereference() const + 0x47 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510ca5b2  : boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> >::reference boost::iterator_core_access::dereference<boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >(boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > const&) + 0x18 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510c61bc  : boost::iterator_facade<boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> >, boost::signals2::detail::void_type, boost::single_pass_traversal_tag, boost::signals2::detail::void_type const&, long>::operator*() const + 0x20 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510c082b  : void boost::signals2::optional_last_value<void>::operator()<boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >(boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> >, boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> >) const + 0x17 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510baf43  : void boost::signals2::detail::combiner_invoker<void>::operator()<boost::signals2::optional_last_value<void>, boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >(boost::signals2::optional_last_value<void>&, boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> >, boost::signals2::detail::slot_call_iterator_t<boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, hpx::parcelset::parcelhandler&, hpx::naming::gid_type>, std::_List_iterator<boost::shared_ptr<boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> > >, boost::signals2::detail::connection_body<std::pair<boost::signals2::detail::slot_meta_group, boost::optional<int> >, boost::signals2::slot<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)> >, hpx::lcos::local::mutex> >) const + 0x63 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510b5429  : boost::signals2::detail::signal_impl<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::signals2::optional_last_value<void>, int, std::less<int>, boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)>, boost::function<void (boost::signals2::connection const&, hpx::parcelset::parcelhandler&, hpx::naming::gid_type)>, hpx::lcos::local::mutex>::operator()(hpx::parcelset::parcelhandler&, hpx::naming::gid_type) + 0x239 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510ade4b  : boost::signals2::signal<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type), boost::signals2::optional_last_value<void>, int, std::less<int>, boost::function<void (hpx::parcelset::parcelhandler&, hpx::naming::gid_type)>, boost::function<void (boost::signals2::connection const&, hpx::parcelset::parcelhandler&, hpx::naming::gid_type)>, hpx::lcos::local::mutex>::operator()(hpx::parcelset::parcelhandler&, hpx::naming::gid_type) + 0x3f in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1510aa27f  : hpx::parcelset::policies::global_parcelhandler_queue::add_parcel(hpx::parcelset::parcel const&) + 0x201 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa15126d67b  : hpx::parcelset::parcelhandler::decode_parcel(boost::shared_ptr<std::vector<char, std::allocator<char> > >, hpx::performance_counters::parcels::data_point) + 0x16d in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa151279ef1  : boost::_mfi::mf2<hpx::threads::thread_state_enum, hpx::parcelset::parcelhandler, boost::shared_ptr<std::vector<char, std::allocator<char> > >, hpx::performance_counters::parcels::data_point>::operator()(hpx::parcelset::parcelhandler*, boost::shared_ptr<std::vector<char, std::allocator<char> > >, hpx::performance_counters::parcels::data_point) const + 0xbd in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa151279a28  : hpx::threads::thread_state_enum boost::_bi::list3<boost::_bi::value<hpx::parcelset::parcelhandler*>, boost::_bi::value<boost::shared_ptr<std::vector<char, std::allocator<char> > > >, boost::_bi::value<hpx::performance_counters::parcels::data_point> >::operator()<hpx::threads::thread_state_enum, boost::_mfi::mf2<hpx::threads::thread_state_enum, hpx::parcelset::parcelhandler, boost::shared_ptr<std::vector<char, std::allocator<char> > >, hpx::performance_counters::parcels::data_point>, boost::_bi::list1<hpx::threads::thread_state_ex_enum&> >(boost::_bi::type<hpx::threads::thread_state_enum>, boost::_mfi::mf2<hpx::threads::thread_state_enum, hpx::parcelset::parcelhandler, boost::shared_ptr<std::vector<char, std::allocator<char> > >, hpx::performance_counters::parcels::data_point>&, boost::_bi::list1<hpx::threads::thread_state_ex_enum&>&, long) + 0xb6 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa151279588  : hpx::threads::thread_state_enum boost::_bi::bind_t<hpx::threads::thread_state_enum, boost::_mfi::mf2<hpx::threads::thread_state_enum, hpx::parcelset::parcelhandler, boost::shared_ptr<std::vector<char, std::allocator<char> > >, hpx::performance_counters::parcels::data_point>, boost::_bi::list3<boost::_bi::value<hpx::parcelset::parcelhandler*>, boost::_bi::value<boost::shared_ptr<std::vector<char, std::allocator<char> > > >, boost::_bi::value<hpx::performance_counters::parcels::data_point> > >::operator()<hpx::threads::thread_state_ex_enum>(hpx::threads::thread_state_ex_enum&) + 0x44 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1512791e5  : hpx::util::detail::vtable<false>::type<boost::_bi::bind_t<hpx::threads::thread_state_enum, boost::_mfi::mf2<hpx::threads::thread_state_enum, hpx::parcelset::parcelhandler, boost::shared_ptr<std::vector<char, std::allocator<char> > >, hpx::performance_counters::parcels::data_point>, boost::_bi::list3<boost::_bi::value<hpx::parcelset::parcelhandler*>, boost::_bi::value<boost::shared_ptr<std::vector<char, std::allocator<char> > > >, boost::_bi::value<hpx::performance_counters::parcels::data_point> > >, hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), void, void>::invoke_(void**, hpx::threads::thread_state_ex_enum, mpl_::bool_<false>) + 0x25 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa151278efc  : hpx::util::detail::vtable<false>::type<boost::_bi::bind_t<hpx::threads::thread_state_enum, boost::_mfi::mf2<hpx::threads::thread_state_enum, hpx::parcelset::parcelhandler, boost::shared_ptr<std::vector<char, std::allocator<char> > >, hpx::performance_counters::parcels::data_point>, boost::_bi::list3<boost::_bi::value<hpx::parcelset::parcelhandler*>, boost::_bi::value<boost::shared_ptr<std::vector<char, std::allocator<char> > > >, boost::_bi::value<hpx::performance_counters::parcels::data_point> > >, hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), void, void>::invoke(void**, hpx::threads::thread_state_ex_enum) + 0x23 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1512e2183  : hpx::util::function_base<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), void, void>::operator()(hpx::threads::thread_state_ex_enum) const + 0x5b in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1512e16b0  : boost::coroutines::detail::unpacker_n<boost::coroutines::tuple_traits<hpx::threads::thread_state_ex_enum, boost::tuples::null_type>, 1>::result<hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)> const, boost::tuples::tuple<hpx::threads::thread_state_ex_enum, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type> >::type boost::coroutines::detail::unpacker_n<boost::coroutines::tuple_traits<hpx::threads::thread_state_ex_enum, boost::tuples::null_type>, 1>::operator()<hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)> const, boost::tuples::tuple<hpx::threads::thread_state_ex_enum, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type> >(hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)> const&, boost::tuples::tuple<hpx::threads::thread_state_ex_enum, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type>&) + 0x30 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1512e071d  : boost::coroutines::detail::unpacker_n<boost::coroutines::tuple_traits<hpx::threads::thread_state_ex_enum, boost::tuples::null_type>, boost::coroutines::tuple_traits<hpx::threads::thread_state_ex_enum, boost::tuples::null_type>::length>::result<hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)>, boost::tuples::tuple<hpx::threads::thread_state_ex_enum, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type> >::type boost::coroutines::detail::unpack<boost::coroutines::tuple_traits<hpx::threads::thread_state_ex_enum, boost::tuples::null_type>, hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)>, boost::tuples::tuple<hpx::threads::thread_state_ex_enum, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type> >(hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)> const&, boost::tuples::tuple<hpx::threads::thread_state_ex_enum, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type, boost::tuples::null_type>&, boost::coroutines::detail::trait_tag<boost::coroutines::tuple_traits<hpx::threads::thread_state_ex_enum, boost::tuples::null_type> >) + 0x27 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1512df35f  : boost::disable_if<boost::is_void<hpx::threads::thread_state_enum>, void>::type boost::coroutines::detail::coroutine_impl_wrapper<hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)>, boost::coroutines::coroutine<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), hpx::threads::detail::coroutine_allocator, boost::coroutines::detail::lx::x86_linux_context_impl>, boost::coroutines::detail::lx::x86_linux_context_impl, hpx::threads::detail::coroutine_allocator>::do_call<hpx::threads::thread_state_enum>(boost::coroutines::detail::coroutine_impl_wrapper<hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)>, boost::coroutines::coroutine<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), hpx::threads::detail::coroutine_allocator, boost::coroutines::detail::lx::x86_linux_context_impl>, boost::coroutines::detail::lx::x86_linux_context_impl, hpx::threads::detail::coroutine_allocator>::dummy<1>) + 0x85 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1512dda87  : boost::coroutines::detail::coroutine_impl_wrapper<hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)>, boost::coroutines::coroutine<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), hpx::threads::detail::coroutine_allocator, boost::coroutines::detail::lx::x86_linux_context_impl>, boost::coroutines::detail::lx::x86_linux_context_impl, hpx::threads::detail::coroutine_allocator>::operator()() + 0x5b in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa1512dc308  : void boost::coroutines::detail::lx::trampoline<boost::coroutines::detail::coroutine_impl_wrapper<hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)>, boost::coroutines::coroutine<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), hpx::threads::detail::coroutine_allocator, boost::coroutines::detail::lx::x86_linux_context_impl>, boost::coroutines::detail::lx::x86_linux_context_impl, hpx::threads::detail::coroutine_allocator> >(boost::coroutines::detail::coroutine_impl_wrapper<hpx::util::function_nonser<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum)>, boost::coroutines::coroutine<hpx::threads::thread_state_enum (hpx::threads::thread_state_ex_enum), hpx::threads::detail::coroutine_allocator, boost::coroutines::detail::lx::x86_linux_context_impl>, boost::coroutines::detail::lx::x86_linux_context_impl, hpx::threads::detail::coroutine_allocator>*) + 0x18 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
@hkaiser

This comment has been minimized.

Copy link
Member

commented Aug 11, 2012

What command lines have you been using? How many localities, how many threads on each of those? Have you used a dedicated AGAS server? If yes, what changes have you made to the code to allow having one?

@brycelelbach

This comment has been minimized.

Copy link
Member Author

commented Aug 11, 2012

All that info is in my scripts on hermione, in /home/wash/sandbox/pbs/sheneos and /home/wash/hpx (my source tree). I'm away from the computer ATM, will update the ticket with the relevant information tonight.

On Aug 11, 2012, at 9:48 AM, Hartmut Kaiser notifications@github.com wrote:

What command lines have you been using? How many localities, how many threads on each of those? Have you used a dedicated AGAS server? If yes, what changes have you made to the code to allow having one?


Reply to this email directly or view it on GitHub.

@brycelelbach

This comment has been minimized.

Copy link
Member Author

commented Aug 11, 2012

This is reproducible without any modifications. Using 930877c, I can reproduce this running sheneos with the 6GB table,
--seed=1337, 4 partitions and 4 localities (4 threads each). The build is full debug.

@hkaiser

This comment has been minimized.

Copy link
Member

commented Aug 11, 2012

Please attach the full submission script.

@brycelelbach

This comment has been minimized.

Copy link
Member Author

commented Aug 12, 2012

#! /bin/bash
#
# Copyright (c) 2009-2011 Bryce Lelbach
#
# Distributed under the Boost Software License, Version 1.0. (See accompanying
# file BOOST_LICENSE_1_0.rst or copy at http://www.boost.org/LICENSE_1_0.txt)
#
#PBS -l nodes=4:beowulf:ppn=4,walltime=00:10:00

COUNTERS="--hpx:print-counter=/statistics{/threads{locality#0/total}/idle-rate}/average@100
          --hpx:print-counter=/statistics{/threads{locality#1/total}/idle-rate}/average@100
          --hpx:print-counter=/statistics{/threads{locality#2/total}/idle-rate}/average@100
          --hpx:print-counter=/statistics{/threads{locality#3/total}/idle-rate}/average@100"

APP_PATH=/home/wash/hpx/gcc-4.6.2-debug/bin/sheneos_test
APP_OPTIONS="--file=/opt/hdf5/sheneos_440r_360t_260y_20120427.h5
             --seed=1337
             --num-partitions=4"

pbsdsh -u $APP_PATH --hpx:nodes=`cat $PBS_NODEFILE` $APP_OPTIONS $COUNTERS 
@brycelelbach

This comment has been minimized.

Copy link
Member Author

commented Aug 12, 2012

You can remove --seed and as well as the counters, and it will still reproduce:

#! /bin/bash
#
# Copyright (c) 2009-2011 Bryce Lelbach
#
# Distributed under the Boost Software License, Version 1.0. (See accompanying
# file BOOST_LICENSE_1_0.rst or copy at http://www.boost.org/LICENSE_1_0.txt)
#
#PBS -l nodes=4:beowulf:ppn=4,walltime=00:10:00

APP_PATH=/home/wash/hpx/gcc-4.6.2-debug/bin/sheneos_test
APP_OPTIONS="--file=/opt/hdf5/sheneos_440r_360t_260y_20120427.h5
             --num-partitions=4"

pbsdsh -u $APP_PATH --hpx:nodes=`cat $PBS_NODEFILE` $APP_OPTIONS

Also, it reproduces with gcc-4.4.6-debug, gcc-4-6.2-release and gcc-4.6.2-debug.

@brycelelbach

This comment has been minimized.

Copy link
Member Author

commented Aug 12, 2012

Sometimes I get this too:

stack_trace]: 15 frames:
0x7fa3cd12b0d2  : boost::backtrace::backtrace(unsigned long) + 0x80 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3cd12b236  : boost::trace() + 0x1e in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3cd11f663  : hpx::detail::backtrace() + 0x18 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3cd12e96d  : void hpx::detail::throw_exception<hpx::exception>(hpx::exception const&, std::string const&, std::string const&, long) + 0x3f in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3cd11f8d8  : hpx::detail::assertion_failed_msg(char const*, char const*, char const*, char const*, long) + 0x238 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3cd11f6a0  : hpx::detail::assertion_failed_msg(char const*, char const*, char const*, char const*, long) + 0x0 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0xbbfbd8        : boost::asio::io_service::id::id() + 0x0 in /home/wash/hpx/gcc-4.6.2-debug/bin/sheneos_test
0x7fa3cd41be5e  : hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>::tfunc_impl(unsigned long) + 0x940 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3cd41ad87  : hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>::tfunc(unsigned long) + 0x195 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3cd43702d  : boost::_mfi::mf1<void, hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>, unsigned long>::operator()(hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>*, unsigned long) const + 0x63 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3cd436f0f  : void boost::_bi::list2<boost::_bi::value<hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>*>, boost::_bi::value<unsigned long> >::operator()<boost::_mfi::mf1<void, hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>, unsigned long>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf1<void, hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>, unsigned long>&, boost::_bi::list0&, int) + 0x69 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3cd436c99  : boost::_bi::bind_t<void, boost::_mfi::mf1<void, hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>, unsigned long>, boost::_bi::list2<boost::_bi::value<hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>*>, boost::_bi::value<unsigned long> > >::operator()() + 0x39 in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3cd43657e  : boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf1<void, hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>, unsigned long>, boost::_bi::list2<boost::_bi::value<hpx::threads::threadmanager_impl<hpx::threads::policies::local_priority_queue_scheduler, hpx::threads::policies::callback_notifier>*>, boost::_bi::value<unsigned long> > > >::run() + 0x1e in /home/wash/hpx/gcc-4.6.2-debug/install/lib/hpx/libhpxd.so.1
0x7fa3c897641e  : ??? + 0x7fa3c897641e in /opt/boost/1.48.0-debug/stage/lib/libboost_thread.so.1.48.0
0x7fa3c756fb50  : ??? + 0x7fa3c756fb50 in /lib/x86_64-linux-gnu/libpthread.so.0
[what]: assertion '!scheduler_.get_thread_count(unknown, num_thread)' failed: HPX(assertion_failure)
[locality-id]: 0
[hostname]: 10.1.1.111:7911
[process-id]: 759
[function]: void hpx::threads::threadmanager_impl<SchedulingPolicy, NotificationPolicy>::tfunc_impl(std::size_t) [with SchedulingPolicy = hpx::threads::policies::local_priority_queue_scheduler, NotificationPolicy = hpx::threads::policies::callback_notifier, std::size_t = long unsigned int]
[file]: /home/wash/hpx/src/runtime/threads/threadmanager.cpp
[line]: 1570
[version]: V1.0.0-trunk (AGAS: V2.1)
[boost]: V1.48.0
[build-type]: debug
[date]: Aug 11 2012 03:54:17
[platform]: linux
[compiler]: GNU C++ version 4.6.2
[stdlib]: GNU libstdc++ version 20120120

pbsdsh: task 1 exit status 262

@hkaiser

This comment has been minimized.

Copy link
Member

commented Aug 12, 2012

I'm not able to reproduce this. sheneos_test works reliably on 4 localities with 4 threads each in release and debug for me. Are you sure to work from top of trunk?

@brycelelbach

This comment has been minimized.

Copy link
Member Author

commented Aug 12, 2012

Yes. Did you test on hermione

On Aug 12, 2012, at 6:06 PM, Hartmut Kaiser notifications@github.com wrote:

I'm not able to reproduce this. sheneos_test works reliably on 4 localities with 4 threads each in release and debug for me. Are you sure to work from top of trunk?


Reply to this email directly or view it on GitHub.

@hkaiser

This comment has been minimized.

Copy link
Member

commented Oct 3, 2012

That's resolved by db79854

@hkaiser hkaiser closed this Oct 3, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.