Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

delay fingerprinting of host #7459

Merged
merged 1 commit into from
Oct 24, 2016

Conversation

lsato-r7
Copy link
Contributor

@lsato-r7 lsato-r7 commented Oct 18, 2016

MS-2073

  • imports are slow mainly caused by fingerprinting after every service creation
  • now only fingerprints after all the services are created for imports

OS Fingerprinting was making imports take a while if there were hosts with thousands of hosts. We were fingerprinting as a callback of service.rb and note.rb after every service and note creation. With the metasploit data model changes and these changes, for imports, we mark the workspace 'import_fingerprint' attribute as true not kicking off the callback to normalize the host. Imports that used to take 5 hours + now takes <10 minutes. Yay!

Verification

  • make sure delay fingerprinting during imports metasploit_data_models#163 is merged in metasploit_data_models before continuing
  • Start msfconsole
  • grab a file to import
  • verify db_import /path/to/file/to/import.xml passes
  • verify the hosts imported are fingerprinted
  • verify import still works in pro and hosts are fingerprinted

MS-2073

 * imports are slow mainly caused by fingerprinting after every service creation
 * now only fingerprints after all the services are created for imports
@acammack-r7 acammack-r7 self-assigned this Oct 21, 2016
@acammack-r7 acammack-r7 merged commit f18cbd6 into rapid7:master Oct 24, 2016
acammack-r7 added a commit that referenced this pull request Oct 24, 2016
@acammack-r7
Copy link
Contributor

acammack-r7 commented Oct 24, 2016

Release Notes

This change delays the fingerprinting of imported hosts until after the end of an import, instead of fingerprinting them after every service is saved. The result is a drastic reduction of time importing hosts with many services.

@tdoan-r7 tdoan-r7 added the rn-enhancement release notes enhancement label Nov 28, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rn-enhancement release notes enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants