Persistent connections in BaseHTTPServer

[lordsutch]

BPO	430706
Nosy	@loewis
Files	httpserver.diff: Yet another patch revision.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/loewis'
closed_at = <Date 2002-03-17.18:38:39.000>
created_at = <Date 2001-06-06.15:33:53.000>
labels = ['library']
title = 'Persistent connections in BaseHTTPServer'
updated_at = <Date 2002-03-17.18:38:39.000>
user = 'https://bugs.python.org/lordsutch'

bugs.python.org fields:

activity = <Date 2002-03-17.18:38:39.000>
actor = 'loewis'
assignee = 'loewis'
closed = True
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2001-06-06.15:33:53.000>
creator = 'lordsutch'
dependencies = []
files = ['3352']
hgrepos = []
issue_num = 430706
keywords = ['patch']
message_count = 10.0
messages = ['36732', '36733', '36734', '36735', '36736', '36737', '36738', '36739', '36740', '36741']
nosy_count = 2.0
nosy_names = ['loewis', 'lordsutch']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue430706'
versions = []

[lordsutch]

This patch provides HTTP/1.1 persistent
connection support in BaseHTTPServer.py. It is
not enabled by default (for backwards
compatibility) because Content-Length headers
must be supplied for persistent connections to
work correctly.

[loewis]

Logged In: YES
user_id=21627

I haven't studied the patch in detail, yet, but I have a
few comments on the style:

• there is no need to quote all authors of the RFC. Also,
  the reference to long-ago expired HTTP draft should go;
  just replace it with a single reference to the RFC
  number (giving an URL for the RFC might be convenient)
• Where is the documentation? A patch to
  Doc/lib/libbasehttp.tex would be appreciated. If you don't
  speak TeX, don't worry: Just write plain text, we'll do
  the mark-up.

[lordsutch]

Logged In: YES
user_id=6757

I have updated the patch against current CVS and have added
documentation.

[loewis]

Logged In: YES
user_id=21627

The patch in its current form seems to be broken. To see the
problem, please run SimpleHTTPServer on some directory, then
access it with a HTTP/1.1 client (e.g. Netscape 4.7).

The server will use the protocol version HTTP/1.0, but the
client will initially send 1.1, and send a Connection:
Keep-alive header. As a result, self.close_connection is set
to 0, despite using HTTP/1.0. In turn, the HTTP server won't
send a content length, and won't close the connection
either. Netscape waits forever from some completion which
never occurs, since the server waits for the next request on
the same connection.

It might be useful to enhance the SimpleHTTPServer test()
function to optionally operate in HTTP/1.1 mode (including
sending a proper ContentLength). Doing the same for the CGI
HTTP server is probably less useful.

[lordsutch]

Logged In: YES
user_id=6757

I reworked the patch a bit to ensure HTTP 1.1 mode is only
used if the handler class is in HTTP 1.1 mode, and
modified the test() functions in the server classes to add
a "protocol" option. I also modified SimpleHTTPServer to
send Content-Length headers for the implemented classes.

[loewis]

Logged In: YES
user_id=21627

It still doesn't work right. If I access SimpleHTTPServer
from a Netscape client, I get error messages like

localhost - - [18/Sep/2001 18:32:22] code 400, message Bad
request syntax ('')
localhost - - [18/Sep/2001 18:32:22] "" 400 -

These are caused because the client closes the connection
after the first request (likely, after it finds out that the
document it got contains no references to the same server
anymore). However, the server continues to invoke
handle_one_request, which reads the empty line and fails to
recognize that the client has closed the connection.

[lordsutch]

Logged In: YES
user_id=6757

I've tracked that one down and will have an updated patch in
a day or two... basically it just needs another else
condition to handle the empty readline(). There are also
some issues for subclasses that probably need to be
documented to play nicely with bad clients like wget that
claim to be HTTP 1.0 but do HTTP 1.1 stuff.

[loewis]

Logged In: YES
user_id=21627

Any chance that an updated patch is forthcoming?

[lordsutch]

Logged In: YES
user_id=6757

Here's my current version of the patch; the main change is that errors now result in closing the connection. A cleaner approach for HTTP 1.1 would be to use Chunked Transfer Encoding for this, so the connection could remain available.

I still get spurious IOErrors (due to SIGPIPEs) that result from clients closing connections. I believe this is because a lot of clients aren't well-behaved; i.e. they read the HTTP/1.1 response line then close the connection immediately. Using TCP_CORK on Linux for sockets might help there, but it's not a general solution. Also, I'm not really sure if these exceptions should be caught here or just left to subclasses to deal with...

[loewis]

Logged In: YES
user_id=21627

Thanks for the patch. Applied as aseHTTPServer.py 1.19,
SimpleHTTPServer.py 1.18, libbasehttp.tex 1.14, NEWS 1.364.