[4.0] hardening resource monitor manager plugin shutdown handling #1774
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
heifner
requested changes
Oct 13, 2023
plugins/resource_monitor_plugin/include/eosio/resource_monitor_plugin/file_space_handler.hpp
Outdated
Show resolved
Hide resolved
plugins/resource_monitor_plugin/include/eosio/resource_monitor_plugin/file_space_handler.hpp
Outdated
Show resolved
Hide resolved
plugins/resource_monitor_plugin/include/eosio/resource_monitor_plugin/file_space_handler.hpp
Outdated
Show resolved
Hide resolved
Those tests were intended to verify the duration of space_monitor_loop. In essence they tested Boost's expires_from_noa, which was not necessary. The tests themselves were hacky and took uncessary 50 seconds.
* removes unnecessary check of info logging "Creating and st arting monitor thread" * reduces time to wait for nodeos startup from 120 seconds to 10 seconds
heifner
requested changes
Oct 16, 2023
plugins/resource_monitor_plugin/include/eosio/resource_monitor_plugin/file_space_handler.hpp
Outdated
Show resolved
Hide resolved
spoonincode
reviewed
Oct 16, 2023
| boost::system::error_code ec; | ||
| timer.cancel(ec); | ||
| } | ||
| thread_pool.stop(); |
There was a problem hiding this comment.
Once stop() returns it's impossible for timer to be accessed from the thread_pool, so I'm not immediately seeing a need this mutex: just stop the thread_pool and let timer be dtor'ed on main thread as it is already.
If you do that, I also suspect you don't even need
because once the
thread_pool is stopped no callbacks will be run, ergo the cancellation callback of the timer in its dtor will never be run.
There was a problem hiding this comment.
Thanks for your insight. I updated the code.
* no need to call cancel timer explicitly. * no need to use mutex for timer as when timer is used on the main thread, resource monitor thread has already stopped.
spoonincode
approved these changes
Oct 16, 2023
heifner
approved these changes
Oct 16, 2023
This was referenced Oct 17, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#1485 reports occasional resource manager plugin test failures. The failure is shown as
/__w/leap/leap/plugins/resource_monitor_plugin/test/test_resmon_plugin.cpp(147): fatal error: in "resmon_plugin_tests/startupNormal": unexpected exception thrown by plugin_startup({"/tmp"})without any other additional information.
Further investigation reveals the scheduled monitor timer task is not cancelled during plugin shutdown and currently monitor task might be still running before the thread exits. This PR hardens the plugin shutdown process.
Resolves #1485