Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stacktrace on SIGSEGV #1985

Merged
merged 2 commits into from Sep 24, 2019
Merged

Add stacktrace on SIGSEGV #1985

merged 2 commits into from Sep 24, 2019

Conversation

jmjatlanta
Copy link
Contributor

@jmjatlanta jmjatlanta commented Sep 11, 2019

This will enable a stacktrace dump when the SIGSEGV signal is raised.

The stacktrace will be written to the log if Boost is version 1.65 or greater, and the OS is Linux or Windows. Nothing will be written to the log if the Boost version is below 1.65.

To make the output human-legible, compile with at least some debug information (i.e. RelWithDebInfo or Debug)

@pmconrad
Copy link
Contributor

Also in cli_wallet?

@jmjatlanta
Copy link
Contributor Author

Also in cli_wallet?

Good question. I hesitated, as I assumed a crash there is a bit easier to figure out, and I didn't want the real problem scrolling off the screen. It is a one-liner to add it. If you think it should be there, I'll be happy to include it. Just say the word.

@pmconrad
Copy link
Contributor

Hm. Please?

I think the real cause of a SIGSEGV will rarely show itself. Stacktrace will usually be more helpful than whatever scrolls off-screen.

@abitmore abitmore added this to In development in Protocol Upgrade Release (5.0.0) via automation Sep 11, 2019
@abitmore abitmore added this to the 4.1.0 - Feature Release milestone Sep 11, 2019
Protocol Upgrade Release (5.0.0) automation moved this from In development to In testing Sep 12, 2019
pmconrad
pmconrad previously approved these changes Sep 12, 2019
Copy link
Contributor

@pmconrad pmconrad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Provoked a null-pointer dereference in account_history plugin, with this result:

1188407ms th_a       application.cpp:392           operator()           ] Initializing database...
1195260ms th_a       db_management.cpp:65          reindex              ] reindexing blockchain
1195260ms th_a       db_management.cpp:70          reindex              ] Replaying blocks, starting at 1...
1195392ms th_a       stacktrace.cpp:24             segfault_signal_hand ]  0# 0x00000000015470DC in graphene/programs/witness_node/witness_node
 1# 0x00007FE4707FF1E0 in /lib64/libc.so.6
 2# 0x0000000001134837 in graphene/programs/witness_node/witness_node
 3# graphene::account_history::detail::account_history_plugin_impl::update_account_histories(graphene::protocol::signed_block const&) in graphene/programs/witness_node/witness_node
 4# boost::signals2::detail::signal_impl<void (graphene::protocol::signed_block const&), boost::signals2::optional_last_value<void>, int, std::less<int>, boost::function<void (graphene::protocol::signed_block const&)>, boost::function<void (boost::signals2::connection const&, graphene::protocol::signed_block const&)>, boost::signals2::mutex>::operator()(graphene::protocol::signed_block const&) in graphene/programs/witness_node/witness_node
 5# graphene::chain::database::notify_applied_block(graphene::protocol::signed_block const&) in graphene/programs/witness_node/witness_node
 6# graphene::chain::database::_apply_block(graphene::protocol::signed_block const&) in graphene/programs/witness_node/witness_node
 7# graphene::chain::database::apply_block(graphene::protocol::signed_block const&, unsigned int) in graphene/programs/witness_node/witness_node
 8# graphene::chain::database::reindex(fc::path) in graphene/programs/witness_node/witness_node
 9# graphene::chain::database::open(fc::path const&, std::function<graphene::chain::genesis_state_type ()>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) in graphene/programs/witness_node/witness_node
10# graphene::app::detail::application_impl::startup() in graphene/programs/witness_node/witness_node
11# graphene::app::application::startup() in graphene/programs/witness_node/witness_node
12# main in graphene/programs/witness_node/witness_node
13# __libc_start_main in /lib64/libc.so.6
14# _start in graphene/programs/witness_node/witness_node

(This was a RelWithDebInfo build. Interestingly the output looks the same even after stripping the executable.)

@abitmore
Copy link
Member

What will the output look like if compiled with Release?

@jmjatlanta
Copy link
Contributor Author

I am surprised to see that @pmconrad is correct. I compiled in Release, and received the following by raising SIGSEGV in node.cpp:

3025918ms p2p        stacktrace.cpp:24             segfault_signal_hand ]  0# 0x0000563B1B7ABB9C in ./programs/witness_node/witness_node
 1# 0x00007FC85C1A9F20 in /lib/x86_64-linux-gnu/libc.so.6
 2# raise in /lib/x86_64-linux-gnu/libpthread.so.0
 3# graphene::net::detail::node_impl::p2p_network_connect_loop() in ./programs/witness_node/witness_node
 4# 0x0000563B1B8FA40C in ./programs/witness_node/witness_node
 5# fc::task_base::run_impl() in ./programs/witness_node/witness_node
 6# fc::thread_d::process_tasks() in ./programs/witness_node/witness_node
 7# fc::thread_d::start_process_tasks(boost::context::detail::transfer_t) in ./programs/witness_node/witness_node
 8# make_fcontext in ./programs/witness_node/witness_node

@abitmore
Copy link
Member

What's the conclusion?

@pmconrad
Copy link
Contributor

I think this works better than expected.

@abitmore abitmore changed the title Add stacktrace on SIGSEV Add stacktrace on SIGSEGV Sep 13, 2019
@abitmore
Copy link
Member

There is a typo in commit history (SIGSEV->SIGSEGV), @jmjatlanta I guess it's easy to fix?

Copy link
Member

@abitmore abitmore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix typo in commit history.

Protocol Upgrade Release (5.0.0) automation moved this from In testing to In development Sep 13, 2019
@pmconrad
Copy link
Contributor

@abitmore please re-review.

Protocol Upgrade Release (5.0.0) automation moved this from In development to In testing Sep 24, 2019
@abitmore abitmore merged commit 5c76c5b into develop Sep 24, 2019
Protocol Upgrade Release (5.0.0) automation moved this from In testing to Done Sep 24, 2019
@abitmore abitmore deleted the jmj_stacktrace branch September 24, 2019 10:31
@pmconrad pmconrad added this to In development in Protocol Upgrade Release (4.0.0) via automation Oct 15, 2019
@pmconrad pmconrad moved this from In development to Done in Protocol Upgrade Release (4.0.0) Oct 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

3 participants