Add bootsnap for bootup performance

[adfoster-r7]

Adds caching to Ruby's load path to improve the bootup performance of msfconsole on startup on macOS from ~13 seconds to ~9.7 seconds

Context

I noticed when running strace to dump out the Linux system calls on msfconsole bootup, that when msfconsole attempts to load a Ruby file such as with require 'foo' you can see Ruby attempting to find the source code on disk in multiple places:

openat(... gem1/foo.rb ...) ENOENT (No such file or directory)
openat(... gem2/foo.rb ...) ENOENT (No such file or directory)
openat(... gem3/foo.rb ...) ENOENT (No such file or directory)
openat(... gem4/foo.rb ...) ENOENT (No such file or directory)
... etc ...

Kali example:

Specifically Ruby is checking the global $LOAD_PATH array which tracks the location of all the loaded gems/libraries that Metasploit depends on to find the required Ruby file. Since Metasploit's $LOAD_PATH is pretty big from the amount of gems we have, attempting to load a single Ruby file involves a lot of expensive trial and error until it finds the right file on disk:

>> $LOAD_PATH.length
=> 236

Example values of $LOAD_PATH

>> $LOAD_PATH
=> 
["/Users/user/Documents/code/metasploit-framework/lib",
 "/Users/user/Documents/code/metasploit-framework/app/models",
 "/Users/user/Documents/code/metasploit-framework/app/concerns",
 "/Users/user/Documents/code/metasploit-framework/app/validators",
 "/Users/user/.rvm/gems/ruby-3.0.5@metasploit-framework/gems/metasploit-credential-6.0.2/app/concerns",
  ... etc ...
]

Bootsnap can be used to optimize this look up by performing Path Pre-Scanning - i.e. Kernel#require and Kernel#load are modified to eliminate $LOAD_PATH scans

Verification

• Code review
• Confirm the before/after time for:
  • mac - Before: ~13, after ~9.7
  • windows - Before: 28.287, After: 26.2
  • kali - Before: 11.23, after: 9.359
  • pro -
• Confirm there's no issues the readline $LOAD_PATH hack here:

  metasploit-framework/lib/msf/ui/console/driver.rb

  Lines 661 to 675 in 192af21

  	if opts['RealReadline']
  	# Remove the gem version from load path to be sure we're getting the
  	# stdlib readline.
  	gem_dir = Gem::Specification.find_all_by_name('rb-readline').first.gem_dir
  	rb_readline_path = File.join(gem_dir, "lib")
  	index = $LOAD_PATH.index(rb_readline_path)
  	# Bundler guarantees that the gem will be there, so it should be safe to
  	# assume we found it in the load path, but check to be on the safe side.
  	if index
  	$LOAD_PATH.delete_at(index)
  	end
  	end
  	begin
  	require 'readline'

  • Both bundle exec ruby ./msfconsole -q --real-readline and bundle exec ruby ./msfconsole -q --no-readline appear to work as expected

[adfoster-r7]

Metadata

$ cat ~/.msf4/bootsnap_cache/metadata.yaml
---
metasploit_framework_version: 6.3.9
ruby_description: ruby 3.0.5p211 (2022-11-24 revision ba5cf0f7c5) [x86_64-darwin20]
bundler_lockfile_hash: 00453b181bd04b9dfc411f0151514c1b
bootsnap_config:
  load_path_cache: true
  compile_cache_iseq: false
  compile_cache_yaml: false

[dwelch-r7]

Release Notes

Adds caching to Ruby's load path logic to improve the bootup performance of msfconsole on startup, averaging 2-3 seconds faster boot time on the tested hardware