HTTPException.description can no longer be None in werkzeug 2.x.

[mhaas]

The werkzeug HTTPException class has a description property, which is allowed to be None according to type hints.

In werkzeug 1.x, this worked fine. In werkzeug 2.x, having description set to None leads to an exception:

Python 3.7.5 (v3.7.5:5c02a39a0b, Oct 14 2019, 18:49:57) 
[Clang 6.0 (clang-600.0.57)] on darwin
from werkzeug.exceptions import HTTPException
e = HTTPException()
e.get_description()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Users/d073668/work/ml/meh/ML-MEH-Commons/venv/lib/python3.7/site-packages/werkzeug/exceptions.py", line 158, in get_description
    description = escape(self.description).replace("\n", "<br>")  # type: ignore
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/html/__init__.py", line 19, in escape
    s = s.replace("&", "&amp;") # Must be done first!
AttributeError: 'NoneType' object has no attribute 'replace'

I believe this error is happening because the escape function used in HTTPException.get_description has been replaced. In Werkzeug 1.x, this was werkzeug.utils.escape which replaces None with the empty string"". In werkzeug 2.x, this is now coming from Python's built-in html package, which does not handle None.

Environment:

• Python version: 3.7.5
• Werkzeug version: 2.0.0 (broken)

[chandanch]

@mhaas Are you also facing this error:

 File "/usr/local/lib/python3.6/site-packages/werkzeug/utils.py", line 553, in redirect
    display_location = html.escape(location)
  File "/usr/local/lib/python3.6/html/__init__.py", line 19, in escape
    s = s.replace("&", "&") # Must be done first!
AttributeError: 'ResponseFailure' object has no attribute 'replace'

[pgjones]

Looks to be this commit (with relevant line highlighted).

[ThiefMaster]

Again, you need to pin your dependencies, especially when it comes to major version upgrades.

[chandanch]

Not able to pin the dependencies as its configured with the pipeline. In the below, the dependencies are obtained from safety repo: https://github.com/pyupio/safety

Below is how the pipeline takes:

Collecting git+git://github.com/pyupio/safety.git (from -r requirements/dev.txt (line 5))
  Cloning git://github.com/pyupio/safety.git to /tmp/pip-req-build-glv5sl02
  Running command git clone -q git://github.com/pyupio/safety.git /tmp/pip-req-build-glv5sl02
Collecting flake8==3.7.8
  Downloading flake8-3.7.8-py2.py3-none-any.whl (70 kB)
Collecting Werkzeug>=0.7
  Downloading Werkzeug-2.0.0-py3-none-any.whl (288 kB)
Collecting Flask
  Downloading Flask-2.0.0-py3-none-any.whl (93 kB)

Please do let me know how we can pin the dependencies within our pipeline. In the project there is no mention of Werkzeug. Its reading from Safety repo.
I'm trying it but not able to find a way to use 1.0.1. Please do tell me the steps.

[ThiefMaster]

Use something like pip-compile to lock your direct and transitive dependencies.

[chandanch]

Thanks that helped, i specied the dependency explicitly in dev.txt.

[drisspg]

Currently using flask 1.0.2 and would like to update to flask 2.0, we throw aborts with description as type dict which fails for a similar reason seeing as Werkzeug expects it to be of type str. I am curious if there is plans to change that restriction or going forward description has to be a str?

[dianaclarke]

In case it's not obvious, folks following this idiom will run into this error.

• https://flask.palletsprojects.com/en/2.0.x/errorhandling/#generic-exception-handlers

My hacky work around (which is good enough for my toy project):

Before:

def _json_http_errors(e):
    import flask as f

    response = e.get_response()
    data = {"code": e.code, "name": e.name}
    if e.code == 400:
        data["description"] = e.description
    response.data = f.json.dumps(data)
    response.content_type = "application/json"
    return response

After:

def _json_http_errors(e):
    import flask as f
    from werkzeug.wrappers.response import Response as WSGIResponse

    response = WSGIResponse("body omitted", e.code, e.get_headers(None, None))
    data = {"code": e.code, "name": e.name}
    if e.code == 400:
        data["description"] = e.description
    response.data = f.json.dumps(data)
    response.content_type = "application/json"
    return response

PS. The traceback I see is a bit different than the original reporter.

• AttributeError: 'dict' object has no attribute 'replace' (triggered by a call to e.get_response(), where e is a HTTPException)

vs.

• AttributeError: 'NoneType' object has no attribute 'replace' (triggered by a call to e.get_description(), where e is a HTTPException)

Traceback (most recent call last):
  File "/Users/diana/envs/qq/lib/python3.9/site-packages/flask/app.py", line 2051, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/diana/envs/qq/lib/python3.9/site-packages/flask/app.py", line 1501, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/diana/envs/qq/lib/python3.9/site-packages/flask/app.py", line 1369, in handle_user_exception
    return self.handle_http_exception(e)
  File "/Users/diana/envs/qq/lib/python3.9/site-packages/flask/app.py", line 1309, in handle_http_exception
    return self.ensure_sync(handler)(e)
  File "/Users/diana/workspace/conbench/conbench/__init__.py", line 79, in _json_http_errors
    response = e.get_response()
  File "/Users/diana/envs/qq/lib/python3.9/site-packages/werkzeug/exceptions.py", line 202, in get_response
    return WSGIResponse(self.get_body(environ, scope), self.code, headers)
  File "/Users/diana/envs/qq/lib/python3.9/site-packages/werkzeug/exceptions.py", line 171, in get_body
    f"{self.get_description(environ)}\n"
  File "/Users/diana/envs/qq/lib/python3.9/site-packages/werkzeug/exceptions.py", line 158, in get_description
    description = escape(self.description).replace("\n", "<br>")  # type: ignore
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/html/__init__.py", line 19, in escape
    s = s.replace("&", "&amp;") # Must be done first!
AttributeError: 'dict' object has no attribute 'replace'