Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add HttpRawHeaders (file) advanced option to HttpClient #12124

Merged
merged 4 commits into from Jul 26, 2019

Conversation

@wvu-r7
Copy link
Contributor

commented Jul 24, 2019

Is this what you wanted, @terrorbyte?

msf5 auxiliary(scanner/http/title) > run

********************
####################
# Request:
####################
GET / HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Content-Type: application/x-www-form-urlencoded


####################
# Response:
####################
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.7.3
Date: Thu, 25 Jul 2019 02:22:56 GMT
Content-type: text/html; charset=utf-8
Content-Length: 297

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Directory listing for /</title>
</head>
<body>
<h1>Directory listing for /</h1>
<hr>
<ul>
</ul>
<hr>
</body>
</html>

[+] [127.0.0.1:8080] [C:200] [R:] [S:SimpleHTTP/0.6 Python/3.7.3] Directory listing for /
[*] Scanned 1 of 1 hosts (100% complete)
[*] Auxiliary module execution completed
msf5 auxiliary(scanner/http/title) > grep HttpRawHeaders advanced
   HttpRawHeaders                                                           no        Path to ERB-templatized raw headers to append to existing headers
msf5 auxiliary(scanner/http/title) > set httprawheaders headers.txt
httprawheaders => headers.txt
msf5 auxiliary(scanner/http/title) > cat headers.txt
[*] exec: cat headers.txt

X-Files: <%= 'Yeeting into an Area 51 near you on September 20th' %>
msf5 auxiliary(scanner/http/title) > run

********************
####################
# Request:
####################
GET / HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Content-Type: application/x-www-form-urlencoded
X-Files: Yeeting into an Area 51 near you on September 20th


####################
# Response:
####################
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.7.3
Date: Thu, 25 Jul 2019 02:23:01 GMT
Content-type: text/html; charset=utf-8
Content-Length: 297

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Directory listing for /</title>
</head>
<body>
<h1>Directory listing for /</h1>
<hr>
<ul>
</ul>
<hr>
</body>
</html>

[+] [127.0.0.1:8080] [C:200] [R:] [S:SimpleHTTP/0.6 Python/3.7.3] Directory listing for /
[*] Scanned 1 of 1 hosts (100% complete)
[*] Auxiliary module execution completed
msf5 auxiliary(scanner/http/title) >

@wvu-r7 wvu-r7 force-pushed the wvu-r7:feature/http branch from 13c7938 to 8007376 Jul 24, 2019

@wvu-r7 wvu-r7 changed the title Add HttpHeaders file advanced option to HttpClient Add HttpRawHeaders file advanced option to HttpClient Jul 24, 2019

@wvu-r7 wvu-r7 changed the title Add HttpRawHeaders file advanced option to HttpClient Add HttpRawHeaders (input file) advanced option to HttpClient Jul 24, 2019

@wvu-r7

This comment has been minimized.

Copy link
Contributor Author

commented Jul 24, 2019

PR words and example updated to reflect new option name HttpRawHeaders.

@terrorbyte

This comment has been minimized.

Copy link
Contributor

commented Jul 25, 2019

This is a lot closer to what I was looking for. I haven't played with your PR yet, but a few things to note:

  • Argument order: e.g. does the UserAgent option override this new option or vice versa, if so it should be documented or "obvious"
  • Does this apply to all the request options: (cgi, raw, etc)?
  • Would templatizing be possible: I feel like that if we are using the file options intead of a "array" we might as well take advantage of the ability to templatize the input file and allow people to inject variables that might be useful. This could be future-ish, but it was a suprisingly useful behavior I've bumped into recently (GoPhish and friends)
@wvu-r7

This comment has been minimized.

Copy link
Contributor Author

commented Jul 25, 2019

  1. These are raw headers appended after all the others, so there may be duplicates.
  2. It should.
  3. I could make it ERB?
@terrorbyte

This comment has been minimized.

Copy link
Contributor

commented Jul 25, 2019

  1. Excellent, I just wanted to clarify. Especially since issues-as-documentation-as-code.
  2. I figured.
  3. The only real thing is that I would personally prefer to have ERB in order to facilitate anything automatic I would have to do. But, I could also see having to deal with that layer of indirection could be annoying, my personal preference is probably what is ERB, but I don't what to get in the way of "standard"
@wvu-r7

This comment has been minimized.

Copy link
Contributor Author

commented Jul 25, 2019

Pushed ERB templatization. See PR description for updated example.

@wvu-r7 wvu-r7 changed the title Add HttpRawHeaders (input file) advanced option to HttpClient Add HttpRawHeaders (file) advanced option to HttpClient Jul 25, 2019

@asoto-r7

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2019

I got an error with a few AWS gems when I checked out this PR, but I'm not sure why. Are these related?

Could not find aws-partitions-1.189.0 in any of the sources
Run bundle install to install missing gems.

Looking good! Here are my testing steps thus far:

  • Test with raw text and send_request_raw
  • Test with raw text including extra newlines, and send_request_raw
  • Test with ERB for loop and send_request_raw
  • Test with raw text and send_request_cgi

EDIT: That aws stuff has to be a fluke from #11977.

@asoto-r7

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2019

So, I haven't dabbled with ERB in a while, but I'm adding a note here for future friends, since there are some gotchas here. You need to be super careful with newlines in ERB, since HTTP will interpret two of them together as the end of the request. Here's a working example:

msf5 > use auxiliary/scanner/http/http_header 
msf5 auxiliary(scanner/http/http_header) > set RHOST 127.0.0.1
msf5 auxiliary(scanner/http/http_header) > set HttpRawHeaders /my_headers
msf5 auxiliary(scanner/http/http_header) > cat /my_headers
Host: www.metasploit.com<% for i in 0..5 do %>
User: asoto<% end %>
Token: letmein
msf5 auxiliary(scanner/http/http_header) > run

And here's what it'll return:

Host: 127.0.0.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Content-Type: application/x-www-form-urlencoded
Content-Length: 0
Host: www.metasploit.com
User: asoto
User: asoto
User: asoto
User: asoto
User: asoto
User: asoto
Hacking: true

@asoto-r7

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2019

Just a note that we might want to add documentation to the wiki: https://github.com/rapid7/metasploit-framework/wiki/How-to-Send-an-HTTP-Request-Using-HTTPClient

@asoto-r7

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2019

Oh, and, hi @terrorbyte! 😄

@asoto-r7 asoto-r7 merged commit a22ad9a into rapid7:master Jul 26, 2019

3 checks passed

Metasploit Automation - Sanity Test Execution Successfully completed all tests.
Details
Metasploit Automation - Test Execution Successfully completed all tests.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

asoto-r7 added a commit that referenced this pull request Jul 26, 2019

@asoto-r7

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2019

Release Notes

A new HttpRawHeaders option allows arbitrary HTTP headers to be injected into requests processed by send_request_cgi and send_request_raw, including support for Embedded Ruby (ERB) templates.

msjenkins-r7 added a commit that referenced this pull request Jul 26, 2019

@wvu-r7 wvu-r7 deleted the wvu-r7:feature/http branch Jul 26, 2019

@wvu-r7

This comment has been minimized.

Copy link
Contributor Author

commented Jul 26, 2019

Thanks, @asoto-r7!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.