Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Processing of malformed HTTP header names #573

nnposter opened this issue Oct 23, 2016 · 0 comments

Processing of malformed HTTP header names #573

nnposter opened this issue Oct 23, 2016 · 0 comments


Copy link

nnposter commented Oct 23, 2016

Local function parse_header() in library http.lua performs processing of HTTP response headers. It currently returns with an error (and no other results) if a header name is malformed, resulting in a completely failed response. This can be caused by an invalid character in the name or just superfluous whitespace between the name and the colon. This rigidity is causing problems with testing not-so-compliant targets.

A real-world example on which NSE scripts fail:

HTTP/1.1 200 OK
Server: Netgear
Content-Type: text/html
Pragma: no-cache
Last Modified: Fri, 16 July 2001 01:01:01 GMT
Connection: close

In contrast, browsers and other tools tend to handle this situation more gracefully:

  • Firefox ignores the particular header, continuing with the next header.
  • Internet Explorer tries to salvage the name if possible. As an example, it will treat line aaa bbb: ccc as header aaa bbb, despite the illegal space in the middle.
  • Wireshark will treat the remainder of the response as the body.

I would like to propose a change in parse_header() to mimic the Firefox behavior. Namely, a header with a malformed name will be skipped, i.e. it will not show up in the header table, but its original textual representation is still preserved in rawheader.

A corresponding patch is pretty straightforward. For the sake of clarity the patch presented here ignores whitespace changes (option -b). A real patch would include proper indentation.

--- ../../svn/nmap/nselib/http.lua  2016-09-27 12:26:44.928032942 -0600
+++ nselib/http.lua 2016-10-27 07:22:15.613209959 -0600
@@ -661,10 +661,15 @@
   while pos <= #header do
     -- Get the field name.
     e, name = get_token(header, pos)
-    if not name or e > #header or string.sub(header, e, e) ~= ":" then
-      return nil, string.format("Can't get header field name at %q", string.sub(header, pos, pos + 30))
+    -- Do not bail out if the header is malformed. Consume the header line
+    -- anyway, getting to the next header, but do not create a new entry in
+    -- the "header" table.
+    if e then
+      if header:sub(e, e) ~= ":" then
+        name = nil
     pos = e + 1
+    end

     -- Skip initial space.
     pos = skip_lws(header, pos)
@@ -680,6 +685,7 @@
       pos = skip_lws(header, pos)

+    if name then
     -- Set it in our table.
     name = string.lower(name)
     local value = table.concat(words, " ")
@@ -699,6 +705,7 @@
         -- Ignore any cookie parsing error
+    end

     -- Next field, or end of string. (If not it's an error.)
     s, e = string.find(header, "^\r?\n", pos)

Please provide feedback on whether this change would cause any issues.

suraj51k pushed a commit to suraj51k/nmap that referenced this issue Jan 31, 2017
…er lines are still captured in the rawheader list but skipped otherwise. Closes nmap#573.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

No branches or pull requests

1 participant