HTTP Crawler: don't expect page object for msg

[sempervictus]

The crawler_process_page method in HttpCrawler assumes that the
page object passed into the method is not nil when formatting the
msg string for printing to console.
Address the assumption with a ternary check leaving the || "ERR"
handling for page.code itself being nil inside the assignment
when page is not nil.

Testing:
Error accessing page undefined method '[]' for nil:NilClass is
no longer being thrown when scanning an odd HTTP service.

[gwillcox-r7]

This actually is a bug in two places in the code as it seems that modules/auxiliary/scanner/http/crawler.rb actually overrides the definition at

metasploit-framework/modules/auxiliary/scanner/http/crawler.rb

Lines 57 to 65 in 024da20

	#
	# The main callback from the crawler, redefines crawler_process_page() as
	# defined by Msf::Auxiliary::HttpCrawler
	#
	# Data we will report:
	# - The path of any URL found by the crawler (web.uri, :path => page.path)
	# - The occurence of any form (web.form :path, :type (get|post|path_info), :params)
	#
	def crawler_process_page(t, page, cnt)

and also has the same bug that is noted in the library.

[gwillcox-r7]

Alright I think the best solution to this would be to add a guard clause. The change you implemented is also valid for checking page.code is not invalid so I'll keep that in.

[gwillcox-r7]

This actually is a bug in two places in the code as it seems that modules/auxiliary/scanner/http/crawler.rb actually overrides the definition at

metasploit-framework/modules/auxiliary/scanner/http/crawler.rb

Lines 57 to 65 in 024da20

	#
	# The main callback from the crawler, redefines crawler_process_page() as
	# defined by Msf::Auxiliary::HttpCrawler
	#
	# Data we will report:
	# - The path of any URL found by the crawler (web.uri, :path => page.path)
	# - The occurence of any form (web.form :path, :type (get|post|path_info), :params)
	#
	def crawler_process_page(t, page, cnt)

and also has the same bug that is noted in the library.

Fixed this in d20fa45

[gwillcox-r7]

Release Notes

A bug has been fixed in the HTTP crawler module and its associated library whereby the code expected an object to be populated when it may not be. This has been fixed with additional validation.