Wsgiref server does not handle closed connections gracefully

[PetterS]

BPO	34547
Nosy	@cjerdonek, @PetterS, @tirkarthi
PRs	bpo-27682: Handle client connection terminations in wsgiref. #9713 bpo-35047: Better error messages in unittest.mock #10090
Superseder	bpo-27682: wsgiref BaseHandler / SimpleHandler can raise additional errors when handling an error

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-11-02.23:08:33.089>
created_at = <Date 2018-08-30.10:17:15.657>
labels = ['3.8', 'type-feature', 'library']
title = 'Wsgiref server does not handle closed connections gracefully'
updated_at = <Date 2018-11-02.23:08:33.077>
user = 'https://github.com/PetterS'

bugs.python.org fields:

activity = <Date 2018-11-02.23:08:33.077>
actor = 'chris.jerdonek'
assignee = 'none'
closed = True
closed_date = <Date 2018-11-02.23:08:33.089>
closer = 'chris.jerdonek'
components = ['Library (Lib)']
creation = <Date 2018-08-30.10:17:15.657>
creator = 'Petter S'
dependencies = []
files = []
hgrepos = []
issue_num = 34547
keywords = ['patch']
message_count = 3.0
messages = ['324375', '327563', '329170']
nosy_count = 3.0
nosy_names = ['chris.jerdonek', 'Petter S', 'xtreak']
pr_nums = ['9713', '10090']
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '27682'
type = 'enhancement'
url = 'https://bugs.python.org/issue34547'
versions = ['Python 3.8']

[PetterS]

The server in the wsgiref module is actually used a lot. For example, it is the server Django uses for development.

A very common thing that happens during Django development is that the web browser closes the connection for some reason. Then very long stack traces appear in the Django logs:

    [30/Aug/2018 12:10:38] "POST /login/ HTTP/1.1" 200 3964
    Traceback (most recent call last):
      File "d:\python37\Lib\wsgiref\handlers.py", line 138, in run
        self.finish_response()
      File "d:\python37\Lib\wsgiref\handlers.py", line 180, in finish_response
        self.write(data)
      File "d:\python37\Lib\wsgiref\handlers.py", line 274, in write
        self.send_headers()
      File "d:\python37\Lib\wsgiref\handlers.py", line 332, in send_headers
        self.send_preamble()
      File "d:\python37\Lib\wsgiref\handlers.py", line 255, in send_preamble
        ('Date: %s\r\n' % format_date_time(time.time())).encode('iso-8859-1')
      File "d:\python37\Lib\wsgiref\handlers.py", line 453, in _write
        result = self.stdout.write(data)
      File "d:\python37\Lib\socketserver.py", line 796, in write
        self._sock.sendall(b)
    ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
    [30/Aug/2018 12:10:38] "POST /login/ HTTP/1.1" 500 59
    ----------------------------------------
    Exception happened during processing of request from ('127.0.0.1', 50112)
    Traceback (most recent call last):
      File "d:\python37\Lib\wsgiref\handlers.py", line 138, in run
        self.finish_response()
      File "d:\python37\Lib\wsgiref\handlers.py", line 180, in finish_response
        self.write(data)
      File "d:\python37\Lib\wsgiref\handlers.py", line 274, in write
        self.send_headers()
      File "d:\python37\Lib\wsgiref\handlers.py", line 332, in send_headers
        self.send_preamble()
      File "d:\python37\Lib\wsgiref\handlers.py", line 255, in send_preamble
        ('Date: %s\r\n' % format_date_time(time.time())).encode('iso-8859-1')
      File "d:\python37\Lib\wsgiref\handlers.py", line 453, in _write
        result = self.stdout.write(data)
      File "d:\python37\Lib\socketserver.py", line 796, in write
        self._sock.sendall(b)
    ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "d:\python37\Lib\wsgiref\handlers.py", line 141, in run
        self.handle_error()
      File "D:\Virtualenvs\ledev-X6wd5Q8f\lib\site-packages\django\core\servers\basehttp.py", line 86, in handle_error
        super().handle_error()
      File "d:\python37\Lib\wsgiref\handlers.py", line 368, in handle_error
        self.finish_response()
      File "d:\python37\Lib\wsgiref\handlers.py", line 180, in finish_response
        self.write(data)
      File "d:\python37\Lib\wsgiref\handlers.py", line 274, in write
        self.send_headers()
      File "d:\python37\Lib\wsgiref\handlers.py", line 331, in send_headers
        if not self.origin_server or self.client_is_modern():
      File "d:\python37\Lib\wsgiref\handlers.py", line 344, in client_is_modern
        return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9'
    TypeError: 'NoneType' object is not subscriptable

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "d:\python37\Lib\socketserver.py", line 647, in process_request_thread
        self.finish_request(request, client_address)
      File "d:\python37\Lib\socketserver.py", line 357, in finish_request
        self.RequestHandlerClass(request, client_address, self)
      File "d:\python37\Lib\socketserver.py", line 717, in __init__
        self.handle()
      File "D:\Virtualenvs\ledev-X6wd5Q8f\lib\site-packages\django\core\servers\basehttp.py", line 154, in handle
        handler.run(self.server.get_app())
      File "d:\python37\Lib\wsgiref\handlers.py", line 144, in run
        self.close()
      File "d:\python37\Lib\wsgiref\simple_server.py", line 35, in close
        self.status.split(' ',1)[0], self.bytes_sent
    AttributeError: 'NoneType' object has no attribute 'split'
    

---

Obviously, this is a bit annoying when developing with Django daily. Since Django simply uses the wsgiref server, I though the best solution was to handle closed connections more gracefully there.

I think the best solution is to simply add another except clause here:

cpython/Lib/wsgiref/handlers.py

Line 142 in e6dac00

except:

. We should catch ConnectionAbortedError here and return from the function.

I am happy to create a PR if this is what we want to do.

[tirkarthi]

Thanks for the report and PR. I think this is the same as bpo-27682. I noticed the same during running make serve locally for documentation and it's worth fixing. I am just wondering if it makes sense to fix the TypeError and AttributeError too. Also I couldn't see any tests for this and I don't know if it's possible to test this scenario.

I will leave it to the reviewer to close this and raise PR against bpo-27682 or continue discussion here.

Thanks again!

[cjerdonek]

Closing this as a duplicate. Please carry over to bpo-27682 any PR's that are still current.