Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update database connection logic to avoid startup crashes #18538

Conversation

adfoster-r7
Copy link
Contributor

@adfoster-r7 adfoster-r7 commented Nov 14, 2023

Relates to #18462
We were able to replicate the above crashes on @zgoldman-r7's environment

Updates the msfconsole database connection logic to avoid startup crashes

Context

After adding logging via #18535 we discovered that a thread that was killing the main pry stream.

Main thread:

┌──(kali㉿kali)-[~/Dev/metasploit-framework]
└─$ ./msfconsole -q
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
/var/lib/gems/3.1.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require': stream closed in another thread (IOError)
        from /var/lib/gems/3.1.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
        from /var/lib/gems/3.1.0/gems/zeitwerk-2.6.12/lib/zeitwerk/kernel.rb:38:in `require'
        from /var/lib/gems/3.1.0/gems/pry-0.14.2/lib/pry.rb:52:in `<top (required)>'
        from /var/lib/gems/3.1.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
        from /var/lib/gems/3.1.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
        from /var/lib/gems/3.1.0/gems/zeitwerk-2.6.12/lib/zeitwerk/kernel.rb:38:in `require'
        from /home/kali/Dev/metasploit-framework/lib/rex/ui/text/shell.rb:126:in `run'
        from /home/kali/Dev/metasploit-framework/lib/metasploit/framework/command/console.rb:54:in `start'
        from /home/kali/Dev/metasploit-framework/lib/metasploit/framework/command/base.rb:82:in `start'
        from ./msfconsole:23:in `<main>'

Culprit thread:

/var/lib/gems/3.1.0/gems/pg-1.5.4/lib/pg/connection.rb:690:in `connect_poll'
/var/lib/gems/3.1.0/gems/pg-1.5.4/lib/pg/connection.rb:690:in `async_connect_or_reset'
/var/lib/gems/3.1.0/gems/pg-1.5.4/lib/pg/connection.rb:824:in `connect_to_hosts'
/var/lib/gems/3.1.0/gems/pg-1.5.4/lib/pg/connection.rb:759:in `new'
/var/lib/gems/3.1.0/gems/pg-1.5.4/lib/pg.rb:63:in `connect'
/var/lib/gems/3.1.0/gems/activerecord-7.0.8/lib/active_record/connection_adapters/postgresql_adapter.rb:78:in `new_client'
/var/lib/gems/3.1.0/gems/activerecord-7.0.8/lib/active_record/connection_adapters/postgresql_adapter.rb:37:in `postgresql_connection'
/var/lib/gems/3.1.0/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:656:in `public_send'
/var/lib/gems/3.1.0/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:656:in `new_connection'
/var/lib/gems/3.1.0/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:700:in `checkout_new_connection'
/var/lib/gems/3.1.0/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:679:in `try_to_checkout_new_connection'
/var/lib/gems/3.1.0/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:640:in `acquire_connection'
/var/lib/gems/3.1.0/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:341:in `checkout'
/var/lib/gems/3.1.0/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:181:in `connection'
/var/lib/gems/3.1.0/gems/activerecord-7.0.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:212:in `with_connection'
/home/kali/Dev/metasploit-framework/lib/msf/core/db_manager/connection.rb:119:in `connection_established?'
/home/kali/Dev/metasploit-framework/lib/metasploit/framework/data_service/proxy/core.rb:134:in `method_missing'
/home/kali/Dev/metasploit-framework/lib/msf/core/payload/windows/encrypted_reverse_tcp.rb:26:in `initialize'
/home/kali/Dev/metasploit-framework/modules/payloads/singles/windows/x64/encrypted_shell_reverse_tcp.rb:18:in `initialize'
/home/kali/Dev/metasploit-framework/lib/msf/core/payload_set.rb:156:in `new'
/home/kali/Dev/metasploit-framework/lib/msf/core/payload_set.rb:156:in `calculate_adapted_single_payload'
/home/kali/Dev/metasploit-framework/lib/msf/core/payload_set.rb:80:in `block (2 levels) in recalculate'
/home/kali/Dev/metasploit-framework/lib/msf/core/payload_set.rb:79:in `each_pair'
/home/kali/Dev/metasploit-framework/lib/msf/core/payload_set.rb:79:in `block in recalculate'
/home/kali/Dev/metasploit-framework/lib/msf/core/payload_set.rb:73:in `each_pair'
/home/kali/Dev/metasploit-framework/lib/msf/core/payload_set.rb:73:in `recalculate'
/home/kali/Dev/metasploit-framework/lib/msf/core/module_manager/cache.rb:136:in `refresh_cache_from_module_files'
/home/kali/Dev/metasploit-framework/lib/msf/ui/console/driver.rb:169:in `block in initialize'
/home/kali/Dev/metasploit-framework/lib/msf/core/thread_manager.rb:105:in `block in spawn'

So potentially there is a bug in postgres to resolve; I believe here:

https://github.com/ged/ruby-pg/blob/2218ebf0b5a6057e74cd4d628e0b20011b8c0aff/ext/pg_connection.c#L106-L115

For now, I believe connection_established? should not be called directly - but .active should be used. connection_established? causes a real connection to the DB which can be avoided, and thus the crash is avoided. I believe the crash is still possible in other scenarios.

Verification

  • Verify msfconsole boots up correctly, and that the encrypted warning still appears:
msf6 > use payload/cmd/windows/http/x64/encrypted_shell/reverse_tcp

[!] A database connection is preferred for this module. If this is not possible, please make sure to take note of the ChachaKey & ChachaNonce options used during generation in order to set them correctly when calling a module handler.
msf6 payload(cmd/windows/http/x64/encrypted_shell/reverse_tcp) > 
  • Connect to the database with msfdb init, Verify msfconsole boots up correctly, and that the encrypted warning no longer appears:
msf6 > use payload/cmd/windows/http/x64/encrypted_shell/reverse_tcp
msf6 payload(cmd/windows/http/x64/encrypted_shell/reverse_tcp) > 

@sjanusz-r7 sjanusz-r7 merged commit 9870d97 into rapid7:master Nov 21, 2023
57 checks passed
@adfoster-r7
Copy link
Contributor Author

Release Notes

Fixes an intermittent stream closed in another thread crash when booting msfconsole

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rn-fix release notes fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants