Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Werkzeug incorrectly handles multiline headers #1080

ngaya-ll opened this issue Mar 10, 2017 · 8 comments

Werkzeug incorrectly handles multiline headers #1080

ngaya-ll opened this issue Mar 10, 2017 · 8 comments


Copy link

According to RFC 2616:

HTTP/1.1 header field values can be folded onto multiple lines if the continuation line begins with a space or horizontal tab. All linear white space, including folding, has the same semantics as SP. A recipient MAY replace any linear white space with a single SP before interpreting the field value or forwarding the message downstream.

However, werkzeug does not accept header values with newlines, even if they abide by this convention.

>>> import werkzeug
>>> werkzeug.Headers().add('foo', 'bar\n baz')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../venv/local/lib/python2.7/site-packages/werkzeug/", line 1136, in add
  File ".../venv/local/lib/python2.7/site-packages/werkzeug/", line 1143, in _validate_value
    raise ValueError('Detected newline in header value.  This is '
ValueError: Detected newline in header value.  This is a potential security problem

Also, this restriction is applied inconsistently.

>>> werkzeug.Headers([('foo', 'bar\n baz')])
Headers([('foo', 'bar\n baz')])

I ran into this issue when trying to write test cases relating to nginx forwarding of client certificates via headers, so there is a real use case for supporting this properly.

Copy link

HTTP headers do not allow newlines. The section you're quoting talks about folding, which ought to remove newlines from the unfolded value. This:


should get unfolded to something like this:

"foo bar"

Apart from that, Werkzeug doesn't parse HTTP at this level, this is the job of the WSGI server. The only reason it rejects newlines when parsing requests is to catch security issues.

Copy link

ngaya-ll commented Mar 13, 2017

This ticket was motivated by the real-life behavior of Flask in development mode behind an nginx proxy forwarding client certs. With that setup, I observed newlines in the headers being passed to the application. But when I attempted to replicate this in a unit test I got the above ValueError when building the request headers.

I did a bit more research on this issue and found the following:

  • The HTTP spec (RFC 2616) states that newlines in headers MAY be replaced with a single space, not that they are guaranteed to be (see quote above).

  • The CGI spec (RFC 3875), which WSGI extends, requires newlines in request headers to be replaced:

    Similarly, a header field that spans multiple lines MUST be merged onto a single line.

  • The WSGI spec (PEP 333) also prohibits newlines in response headers:

    Each header_value must not include any control characters, including carriage returns or linefeeds, either embedded or at the end.

So this means that there are two bugs here:

  • The werkzeug development server does not correctly normalize folded strings in request headers.

  • The Headers object is inconsistent about accepting newlines in header values. Newlines are accepted in the constructor, but not in the add() method. It's probably better for the Headers object to remain permissive and perform the validation when actually constructing the WSGI response in BaseResponse. That way forbidding newlines in the constructor won't break compatibility with possibly non-conforming WSGI servers (such as werkzeug's own development server).

Copy link

Fair enough. There's also #1070 which plays into this.

@untitaker untitaker reopened this Mar 14, 2017
Copy link

Fairly sure this was fixed with the fix for #1070. If not, please let me know with a reproducible example.

Copy link

ngaya-ll commented Jan 21, 2019

@davidism As I mentioned in a previous comment, there are actually two bugs here, neither of which has been fixed on the current master branch.

The first bug involves how the werkzeug development server handles line-wrapped headers. It can be reproduced with the following server code, which prints the value of the X-Example header:

from werkzeug.serving import run_simple
from werkzeug.wrappers import Request, Response

def app(environ, start_response):
    request = Request(environ)
    response = Response(status=204)
    return response(environ, start_response)

run_simple('localhost', 8080, app)

We can then send it a request with a header spanning multiple lines:

GET / HTTP/1.1
Host: localhost:8080
Connection: close
X-Example: foo

Expected server output:
Header value is merged onto a single line

'foo bar'

Actual server output (Python 2):

Exception happened during processing of request from ('', 57361)
Traceback (most recent call last):
  File "/usr/lib/python2.7/", line 295, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python2.7/", line 321, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.7/", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.7/", line 649, in __init__
  File "/home/.../venv/local/lib/python2.7/site-packages/werkzeug/", line 320, in handle
    rv = BaseHTTPRequestHandler.handle(self)
  File "/usr/lib/python2.7/", line 340, in handle
  File "/home/.../venv/local/lib/python2.7/site-packages/werkzeug/", line 355, in handle_one_request
    return self.run_wsgi()
  File "/home/.../venv/local/lib/python2.7/site-packages/werkzeug/", line 238, in run_wsgi
    self.environ = environ = self.make_environ()
  File "/home/.../venv/local/lib/python2.7/site-packages/werkzeug/", line 217, in make_environ
    for key, value in self.get_header_items():
  File "/home/.../venv/local/lib/python2.7/site-packages/werkzeug/", line 441, in get_header_items
    key, value = header[0:-2].split(":", 1)
ValueError: need more than 1 value to unpack

Actual server output (Python 3):
Header value contains newline characters disallowed by WSGI spec

'foo\r\n bar'

The second bug has to do with how the Headers object handles newline values.

>>> from werkzeug import Headers
>>> h1 = Headers([('X-Example', 'foo\r\n bar')])
>>> h2 = Headers()
>>> h2.add('X-Example', 'foo\r\n bar')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/.../venv3/lib/python3.6/site-packages/werkzeug/", line 1166, in add
  File "/home/.../venv3/lib/python3.6/site-packages/werkzeug/", line 1173, in _validate_value
    raise ValueError('Detected newline in header value.  This is '
ValueError: Detected newline in header value.  This is a potential security problem

Expected result:
Both operations should have the same result, whether a ValueError or success.

Actual result:
Headers constructor allows newlines while Headers.add() raises ValueError.

Copy link

bertomaniac commented Mar 27, 2019

I was seeing the ValueError recently in one of our projects (Python 2.7 with Flask) which happens to sit behind an nginx proxy. I ended up reverting to 0.14.1 (overriding the version bundled with Flask) and my error went away. Thought I'd add since it seems the 0.15.x branch introduced this problem (or potentially created a new problem with how it was handling request headers).


I tracked down what headers we were passing and one of them was a multi-line cert in pem format i.e.:

    -----END CERTIFICATE-----

Our nginx server was configured like so:

proxy_set_header SSL_CLIENT_CERT $ssl_client_cert;

We should probably be using $ssl_client_escaped_cert instead of $ssl_client_cert (since this is deprecated anyway). Not sure if that change will fix the header parsing problem though.

Hoping this helps anyone else that is running into this problem. It appears that 0.15.1 doesn't properly handle a multi-line header like a PEM cert at this time with Python 2.7.

Copy link

This is a 2.7 issue again caused by the header processing of the development server. I am adding the ability to process folding of headers to the 2.7 compatibility code for request headers.

Copy link

Confirmed that development server bug is now fixed in both Python 2 and 3. The second issue with the Headers object is still present. Opened #1608.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet

No branches or pull requests

5 participants