Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disconnection could cause server abortion. #2216

Closed
o01eg opened this Issue Jul 15, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@o01eg
Copy link
Contributor

o01eg commented Jul 15, 2018

Environment

  • FreeOrion Version: o01eg@86cd23f based on 9a06376 of 0.4.8
  • Operating System: Ubuntu Linux x64 18.04
  • Fetched as
    • Compiled from source

Description

Core was generated by `/usr/lib/freeorion/freeoriond --resource-dir /usr/share/games/freeorion/default'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007f5ea1ef0801 in __GI_abort () at abort.c:79
#2  0x00007f5ea28e38fb in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007f5ea28e9d3a in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007f5ea28e9d95 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007f5ea28e9fe8 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00005569e1bb620e in std::__throw_bad_weak_ptr () at /usr/include/c++/7/bits/shared_ptr_base.h:83
#7  0x00005569e1bb3105 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_add_ref_lock (this=<optimized out>)
    at /usr/include/c++/7/bits/shared_ptr_base.h:246
#8  std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count (__r=..., this=0x7ffd5b27b808) at /usr/include/c++/7/bits/shared_ptr_base.h:852
#9  std::__shared_ptr<PlayerConnection, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<PlayerConnection, void> (__r=..., this=0x7ffd5b27b800)
    at /usr/include/c++/7/bits/shared_ptr_base.h:1147
#10 std::shared_ptr<PlayerConnection>::shared_ptr<PlayerConnection, void> (__r=..., this=0x7ffd5b27b800) at /usr/include/c++/7/bits/shared_ptr.h:266
#11 std::enable_shared_from_this<PlayerConnection>::shared_from_this (this=<optimized out>) at /usr/include/c++/7/bits/shared_ptr.h:640
#12 PlayerConnection::AsyncErrorHandler (this=<optimized out>, handled_error=..., error=...) at ./network/ServerNetworking.cpp:402
#13 0x00005569e1bb443e in boost::_mfi::mf2<void, PlayerConnection, boost::system::error_code, boost::system::error_code>::operator() (a2=..., a1=..., 
    p=<optimized out>, this=<optimized out>) at /usr/include/boost/bind/mem_fn_template.hpp:280
#14 boost::_bi::list3<boost::_bi::value<PlayerConnection*>, boost::_bi::value<boost::system::error_code>, boost::arg<1> (*)()>::operator()<boost::_mfi::mf2<void, PlayerConnection, boost::system::error_code, boost::system::error_code>, boost::_bi::rrlist1<boost::system::error_code const&> > (
    a=<synthetic pointer>..., f=..., this=0x7ffd5b27b9f0) at /usr/include/boost/bind/bind.hpp:398
#15 boost::_bi::bind_t<void, boost::_mfi::mf2<void, PlayerConnection, boost::system::error_code, boost::system::error_code>, boost::_bi::list3<boost::_bi::value<PlayerConnection*>, boost::_bi::value<boost::system::error_code>, boost::arg<1> (*)()> >::operator()<boost::system::error_code const&> (a1=..., 
    this=0x7ffd5b27b9e0) at /usr/include/boost/bind/bind.hpp:1306
#16 boost::asio::detail::binder1<boost::_bi::bind_t<void, boost::_mfi::mf2<void, PlayerConnection, boost::system::error_code, boost::system::error_code>, boost::_bi::list3<boost::_bi::value<PlayerConnection*>, boost::_bi::value<boost::system::error_code>, boost::arg<1> (*)()> >, boost::system::error_code>::operator() (this=0x7ffd5b27b9e0) at /usr/include/boost/asio/detail/bind_handler.hpp:47
#17 boost::asio::asio_handler_invoke<boost::asio::detail::binder1<boost::_bi::bind_t<void, boost::_mfi::mf2<void, PlayerConnection, boost::system::error_code, boost::system::error_code>, boost::_bi::list3<boost::_bi::value<PlayerConnection*>, boost::_bi::value<boost::system::error_code>, boost::arg<1> (*)()> >, boost::system::error_code> > (function=...) at /usr/include/boost/asio/handler_invoke_hook.hpp:69
#18 boost_asio_handler_invoke_helpers::invoke<boost::asio::detail::binder1<boost::_bi::bind_t<void, boost::_mfi::mf2<void, PlayerConnection, boost::system::error_code, boost::system::error_code>, boost::_bi::list3<boost::_bi::value<PlayerConnection*>, boost::_bi::value<boost::system::error_code>, boost::arg<1> (*)()> >, boost::system::error_code>, boost::_bi::bind_t<void, boost::_mfi::mf2<void, PlayerConnection, boost::system::error_code, boost::system::error_code>, boost::_bi::list3<boost::_bi::value<PlayerConnection*>, boost::_bi::value<boost::system::error_code>, boost::arg<1> (*)()> > > (context=..., function=...)
    at /usr/include/boost/asio/detail/handler_invoke_helpers.hpp:37
#19 boost::asio::detail::wait_handler<boost::_bi::bind_t<void, boost::_mfi::mf2<void, PlayerConnection, boost::system::error_code, boost::system::error_code>, boost::_bi::list3<boost::_bi::value<PlayerConnection*>, boost::_bi::value<boost::system::error_code>, boost::arg<1> (*)()> > >::do_complete (
    owner=0x5569e244fb30, base=<optimized out>) at /usr/include/boost/asio/detail/wait_handler.hpp:70
#20 0x00005569e1cf9eb7 in boost::asio::detail::task_io_service_operation::complete (bytes_transferred=<optimized out>, ec=..., owner=..., 
    this=0x7f5e780d9260) at /usr/include/boost/asio/detail/task_io_service_operation.hpp:38
#21 boost::asio::detail::task_io_service::do_run_one (ec=..., this_thread=..., lock=..., this=0x5569e244fb30)
    at /usr/include/boost/asio/detail/impl/task_io_service.ipp:372
#22 boost::asio::detail::task_io_service::run_one (ec=..., this=0x5569e244fb30) at /usr/include/boost/asio/detail/impl/task_io_service.ipp:170
#23 boost::asio::io_service::run_one (this=0x7ffd5b27bc88) at /usr/include/boost/asio/impl/io_service.ipp:72
#24 ServerApp::Run (this=0x7ffd5b27bc80) at ./server/ServerApp.cpp:347
#25 0x00005569e1b82a1f in main (argc=<optimized out>, argv=<optimized out>) at ./server/dmain.cpp:86
(gdb) f 16
#16 boost::asio::detail::binder1<boost::_bi::bind_t<void, boost::_mfi::mf2<void, PlayerConnection, boost::system::error_code, boost::system::error_code>, boost::_bi::list3<boost::_bi::value<PlayerConnection*>, boost::_bi::value<boost::system::error_code>, boost::arg<1> (*)()> >, boost::system::error_code>::operator() (this=0x7ffd5b27b9e0) at /usr/include/boost/asio/detail/bind_handler.hpp:47
47	    handler_(static_cast<const Arg1&>(arg1_));
(gdb) p arg1_
$8 = {m_val = 125, m_cat = 0x7f5ea2de9100}
(gdb) f 6
#6  0x00005569e1bb620e in std::__throw_bad_weak_ptr () at /usr/include/c++/7/bits/shared_ptr_base.h:83
83	  { _GLIBCXX_THROW_OR_ABORT(bad_weak_ptr()); }
(gdb) l
78	  };
79	
80	  // Substitute for bad_weak_ptr object in the case of -fno-exceptions.
81	  inline void
82	  __throw_bad_weak_ptr()
83	  { _GLIBCXX_THROW_OR_ABORT(bad_weak_ptr()); }

Expected Result

Server correctly process disconnection.

Steps to reproduce

  • Not sure but it looks it happened when many disconnections on public server.
@o01eg

This comment has been minimized.

Copy link
Contributor Author

o01eg commented Dec 20, 2018

Got it again:

(gdb) f 12
#12 PlayerConnection::AsyncErrorHandler (this=0x7fdee401a330, handled_error=..., error=...) at ./network/ServerNetworking.cpp:422
422	    EventSignal(boost::bind(m_disconnected_callback, shared_from_this()));
(gdb) p this->m_ID
$4 = 2

In the log last words was:

...
23:55:10.072274 [debug] FSM : ServerFSM.cpp:960 : (ServerFSM) MPLobby.JoinGame player accepted: xxcc3
23:55:10.072320 [debug] server : ServerFSM.cpp:385 : ServerFSM.EstablishPlayer Assign new player id 3
23:55:10.072649 [debug] server : ServerFSM.cpp:397 : ServerFSM.EstablishPlayer player xxcc3 get cookie: 30e3c3af-f046-4614-b4a8-e4a9f345d33a
23:55:10.096156 [debug] server : Building.cpp:407 : BuildingTypeManager checksum: 8322072
23:55:10.096448 [debug] server : Field.cpp:226 : FieldTypeManager checksum: 8702654
23:55:10.096682 [debug] server : Field.cpp:226 : FieldTypeManager checksum: 4039914
23:55:10.096896 [debug] server : Field.cpp:226 : FieldTypeManager checksum: 6891154
23:55:10.097653 [debug] server : Field.cpp:226 : FieldTypeManager checksum: 5426760
23:55:10.098421 [debug] server : Field.cpp:226 : FieldTypeManager checksum: 4597448
23:55:10.098508 [debug] server : Field.cpp:226 : FieldTypeManager checksum: 7001039
23:55:10.120641 [debug] server : ShipDesign.cpp:766 : HullTypeManager checksum: 3483811
23:55:10.128270 [debug] server : ShipDesign.cpp:265 : PartTypeManager checksum: 9512392
23:55:10.129545 [debug] server : ShipDesign.cpp:1540 : PredefinedShipDesignManager checksum: 877965
23:55:10.245094 [debug] server : Species.cpp:629 : SpeciesManager checksum: 4191383
23:55:10.256407 [debug] server : Tech.cpp:715 : TechManager checksum: 6623856
23:55:19.266369 [debug] FSM : ServerFSM.cpp:737 : MPLobby::react(Disconnection) player id: 3
23:55:19.266402 [debug] FSM : ServerFSM.cpp:738 : Remaining player ids: 
23:55:19.266412 [debug] FSM : ServerFSM.cpp:742 :  ... 2 (xxcc2)
23:55:26.269360 [error] network : ServerNetworking.cpp:407 : PlayerConnection::WriteMessage(): player id = 2 error #104 "Connection reset by peer this = 0x7fdee401a330
23:55:26.269816 [debug] FSM : ServerFSM.cpp:737 : MPLobby::react(Disconnection) player id: 2
23:55:26.269838 [debug] FSM : ServerFSM.cpp:738 : Remaining player ids: 
[log ends here]
@o01eg

This comment has been minimized.

Copy link
Contributor Author

o01eg commented Jan 20, 2019

Catch on version 0bbfe41 (based on 0.4.8) with additional trace messages:

freeoriond.log

2019-01-20 15:13:08.667003 [trace] network : ServerNetworking.cpp:735 : ServerNetworking::DisconnectImpl : disconnecting player 2 this = 0x56085c6f0e70
2019-01-20 15:13:08.667106 [debug] FSM : ServerFSM.cpp:249 : ServerFSM::HandleNonLobbyDisconnection : Lost connection to player #2, named "salvor3".
2019-01-20 15:13:08.667123 [error] FSM : ServerFSM.cpp:268 : Player #2, named "salvor3"quit before empire was eliminated.
2019-01-20 15:13:08.667166 [error] FSM : ServerFSM.cpp:291 : Unable to recover server terminating.
2019-01-20 15:13:08.667193 [debug] FSM : ServerFSM.cpp:625 : (ServerFSM) MPLobby. Fill MPLobby data from the previous game.
2019-01-20 15:13:08.667209 [debug] FSM : ServerFSM.cpp:640 : (ServerFSM) MPLobby. Fill MPLobby player 5
2019-01-20 15:13:08.667265 [debug] FSM : ServerFSM.cpp:640 : (ServerFSM) MPLobby. Fill MPLobby player 1
2019-01-20 15:13:08.667292 [debug] FSM : ServerFSM.cpp:640 : (ServerFSM) MPLobby. Fill MPLobby player 3
2019-01-20 15:13:08.667317 [debug] FSM : ServerFSM.cpp:640 : (ServerFSM) MPLobby. Fill MPLobby player 6
2019-01-20 15:13:08.667570 [trace] network : ServerNetworking.cpp:74 : PlayerConnection destruction player id 2 this = 0x56085c6f0e70
2019-01-20 15:13:12.828802 [trace] network : ServerNetworking.cpp:735 : ServerNetworking::DisconnectImpl : disconnecting player 6 this = 0x56085cc92af0
2019-01-20 15:13:12.828904 [debug] FSM : ServerFSM.cpp:754 : MPLobby::react(Disconnection) player id: 6
2019-01-20 15:13:12.828922 [debug] FSM : ServerFSM.cpp:755 : Remaining player ids: 
2019-01-20 15:13:12.828934 [debug] FSM : ServerFSM.cpp:759 :  ... 5 (Divaliz)
2019-01-20 15:13:12.828946 [debug] FSM : ServerFSM.cpp:759 :  ... 1 (Skaidan)
2019-01-20 15:13:12.828956 [debug] FSM : ServerFSM.cpp:759 :  ... 3 (KeepStay)
2019-01-20 15:13:19.832365 [debug] server : Process.cpp:275 : Process::Impl::Kill calling kill(m_process_id, SIGKILL)
2019-01-20 15:13:19.832525 [debug] server : Process.cpp:277 : Process::Impl::Kill calling waitpid(m_process_id, &status, 0)
2019-01-20 15:13:19.832555 [debug] server : Process.cpp:279 : Process::Impl::Kill done
2019-01-20 15:13:19.833357 [trace] network : ServerNetworking.cpp:74 : PlayerConnection destruction player id 6 this = 0x56085cc92af0
2019-01-20 15:13:19.973022 [trace] network : ServerNetworking.cpp:735 : ServerNetworking::DisconnectImpl : disconnecting player 1 this = 0x56085c9170f0
2019-01-20 15:13:19.973085 [debug] FSM : ServerFSM.cpp:754 : MPLobby::react(Disconnection) player id: 1
2019-01-20 15:13:19.973098 [debug] FSM : ServerFSM.cpp:755 : Remaining player ids: 
2019-01-20 15:13:19.973107 [debug] FSM : ServerFSM.cpp:759 :  ... 5 (Divaliz)
2019-01-20 15:13:19.973115 [debug] FSM : ServerFSM.cpp:759 :  ... 3 (KeepStay)
2019-01-20 15:13:26.975587 [debug] server : Process.cpp:275 : Process::Impl::Kill calling kill(m_process_id, SIGKILL)
2019-01-20 15:13:26.975707 [debug] server : Process.cpp:277 : Process::Impl::Kill calling waitpid(m_process_id, &status, 0)
2019-01-20 15:13:26.975738 [debug] server : Process.cpp:279 : Process::Impl::Kill done
2019-01-20 15:13:26.976300 [trace] network : ServerNetworking.cpp:74 : PlayerConnection destruction player id 1 this = 0x56085c9170f0
2019-01-20 15:13:26.976341 [trace] network : ServerNetworking.cpp:89 : Player connection already down.
2019-01-20 15:13:26.976421 [trace] network : ServerNetworking.cpp:735 : ServerNetworking::DisconnectImpl : disconnecting player 5 this = 0x56085c6f1110
2019-01-20 15:13:26.976441 [debug] FSM : ServerFSM.cpp:754 : MPLobby::react(Disconnection) player id: 5
2019-01-20 15:13:26.976452 [debug] FSM : ServerFSM.cpp:755 : Remaining player ids: 
2019-01-20 15:13:26.976465 [debug] FSM : ServerFSM.cpp:759 :  ... 3 (KeepStay)
2019-01-20 15:13:33.979562 [debug] server : Process.cpp:275 : Process::Impl::Kill calling kill(m_process_id, SIGKILL)
2019-01-20 15:13:33.979671 [debug] server : Process.cpp:277 : Process::Impl::Kill calling waitpid(m_process_id, &status, 0)
2019-01-20 15:13:33.979704 [debug] server : Process.cpp:279 : Process::Impl::Kill done
2019-01-20 15:13:33.980267 [error] network : ServerNetworking.cpp:406 : PlayerConnection::WriteMessage(): player id = 3 error #32 "Broken pipe this = 0x56085c933730
2019-01-20 15:13:33.980902 [trace] network : ServerNetworking.cpp:74 : PlayerConnection destruction player id 5 this = 0x56085c6f1110
2019-01-20 15:13:33.980993 [trace] network : ServerNetworking.cpp:89 : Player connection already down.
2019-01-20 15:13:33.981117 [trace] network : ServerNetworking.cpp:735 : ServerNetworking::DisconnectImpl : disconnecting player 3 this = 0x56085c933730
2019-01-20 15:13:33.981197 [debug] FSM : ServerFSM.cpp:754 : MPLobby::react(Disconnection) player id: 3
2019-01-20 15:13:33.981263 [debug] FSM : ServerFSM.cpp:755 : Remaining player ids: 
2019-01-20 15:13:40.983628 [debug] server : Process.cpp:275 : Process::Impl::Kill calling kill(m_process_id, SIGKILL)
2019-01-20 15:13:40.983824 [debug] server : Process.cpp:277 : Process::Impl::Kill calling waitpid(m_process_id, &status, 0)
2019-01-20 15:13:40.983861 [debug] server : Process.cpp:279 : Process::Impl::Kill done
2019-01-20 15:13:40.983995 [trace] network : ServerNetworking.cpp:74 : PlayerConnection destruction player id 3 this = 0x56085c933730
2019-01-20 15:13:40.984033 [trace] network : ServerNetworking.cpp:89 : Player connection already down.
2019-01-20 15:13:40.984177 [trace] network : ServerNetworking.cpp:417 : PlayerConnection::AsyncErrorHandler: player id = 3 handled error #32 error #125 this = 0x56085c933730

Looks like when calling PlayerConnection::AsyncErrorHandler connection already destructed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.