BaseHTTPRequestHandler, update the protocol version to http 1.1 by default?

[matrixise]

BPO	21224
Nosy	@orsenthil, @vstinner, @vadmium, @matrixise

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-04-14.20:43:52.074>
labels = ['type-feature', 'library']
title = 'BaseHTTPRequestHandler, update the protocol version to http 1.1 by default?'
updated_at = <Date 2015-11-26.02:00:27.689>
user = 'https://github.com/matrixise'

bugs.python.org fields:

activity = <Date 2015-11-26.02:00:27.689>
actor = 'martin.panter'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2014-04-14.20:43:52.074>
creator = 'matrixise'
dependencies = []
files = []
hgrepos = []
issue_num = 21224
keywords = []
message_count = 16.0
messages = ['216206', '216214', '216219', '216767', '217584', '229246', '254159', '254163', '254164', '254183', '254250', '255343', '255345', '255389', '255391', '255398']
nosy_count = 4.0
nosy_names = ['orsenthil', 'vstinner', 'martin.panter', 'matrixise']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue21224'
versions = ['Python 3.5']

[matrixise]

Hi,

With this issue, I would like to ask you to use the last version of the HTTP protocol in the BaseHTTPRequestHandler for Python 3.5.
Because this one uses the version 1.0 of the protocol for the backward-compatibility.

https://docs.python.org/3/library/http.server.html?highlight=protocol_version#http.server.BaseHTTPRequestHandler.protocol_version

When we develop an web app with Flask/Werkzeug or an other tool, the default version of
the protocol is "HTTP/1.0". If we use Gunicorn, the protocol version is HTTP/1.1 and not 1.0, but this one can support the old version.

So, I propose to change the default version to the HTTP/1.1. It seems that the code of the server can handle this version without any problem.

HTTP 1.0 - http://www.ietf.org/rfc/rfc1945.txt - 1996
HTTP 1.1 - http://www.ietf.org/rfc/rfc2616.txt - 1999

In 2014, I think we can move to HTTP 1.1 by default.

Regards,

Stephane

[orsenthil]

It may not just the be the version, but the capabilities. We have ensure that capabilities are met/added before updated the version. Thanks for filing the issue.

[matrixise]

gunicorn has an implementation of the HTTP/1.1 protocol, we can ask to the author of this project if we can use its code and reuse it in the standard library.

[vadmium]

Looking at bpo-430706 and revision 27f36f4bf525, there is concious support for HTTP 1.1 persistent connections. Apparently the 1.0 default is just for backwards compatibility.

[matrixise]

In this case, I suggest than by default, we use the version 1.1 of HTTP
and not 1.0.

We are in 2014, I suppose that all the old http clients use the version
1.1 of the protocol,
otherwise, the user can specify the version 1.0.

[matrixise]

ping

[matrixise]

pong

patience is one key to success ;-)

[vadmium]

What are the advantages of changing the default? Just that the user no longer has to set it manually?

What do you think of the problem mentioned in the documentation? If an existing HTTP 1.0 server that works fine in Python 3.5 were to suddenly have protocol_version="HTTP/1.1" forced by default, it sounds like all its responses may stop working (hang from the client’s POV) if they don’t explicitly close the connection.

One option could be to wrap the “wfile” stream in a chunk encoder, and insert Transfer-Encoding: chunked. I haven’t thought through this much, and it may not work very well because Python’s HTTP server is so low-level. This would basically be a HTTP 1.1 to HTTP 1.0 proxy.

Similarly, I understand HTTP 1.1 requires chunked encoding support for requests, but there is no support for that in the Python implementation. Existing servers accepting e.g. POST uploads would expect a Content-Length header, which would be hard to fake with a compatibility proxy (may need to buffer it all to count the bytes).

Another way might be a deprecation cycle, to force people using 3.6 to set protocol_version.

[vadmium]

Actually RFC 7230 says “A server may reject a request that contains a message body but not a Content-Length by responding with 411 (Length Required)”, so maybe it is only clients that have to support chunked decoding. So I take back my paragraph about POST requests.

[matrixise]

After your remarks, maybe we can close this issue. It's not just a simple modification of a string.

Do you know if we want to support the HTTP 1.1 and 2.0 in the future, directly in CPython and not via an external library (gunicorn, ...)

What do you think ? we close this issue and open an other with "Support of HTTP 1.1 and 2.0" ?

[vadmium]

There is already bpo-23794 discussing HTTP 2. IMO it would be interesting to implement, but I’m not sure how appropriate it would be in Python’s standard library at this stage. But working on it could drive other improvements in the existing HTTP stuff, as Demian hinted.

What do you mean by supporting HTTP 1.1 in the future? The basic support is already there; you just have to manually enable it. Maybe there is scope for making 1.1 easier to use, perhaps with a BaseHTTP11RequestHandler (shorter: Http11Handler) class or something.

[vstinner]

For HTTP 2, I consider that the protocol is too young. I prefer to move fast on a library hosted on PyPI, rather than puting something is the stable and "frozen" stdlib.

[vstinner]

Similarly, I understand HTTP 1.1 requires chunked encoding support for requests, but there is no support for that in the Python implementation.

Hum, I don't understand exactly this issue. I understood that Python only has a "partial" implementation of the HTTP 1.1 protocol for the server side, and maybe even for the client side.

Because of that, I don't think that it's good idea to switch to HTTP 1.1 by default. We have to implement missing features server and client side before.

[vadmium]

Victor, that chunked support for server requests is optional, and not actually required (my quote was wrong; see my message after the one you quoted).

The client already does HTTP 1.1 by default:
>>> http.client.HTTPConnection._http_vsn_str
'HTTP/1.1'

It is already possible to make a working HTTP 1.1 server by setting protocol_version. What things do you think would make the HTTP 1.1 server implementation more complete?

[vstinner]

Sorry, I read the issue very quickly. I don't understand why the
default is not changed.

[vadmium]

Changing it could break existing code written for the HTTP 1.0 behaviour. The main effect mentioned at <https://docs.python.org/dev/library/http.server.html#http.server.BaseHTTPRequestHandler.protocol_version\> is that in HTTP 1.0 mode, a server may send a response without Content-Length, because its connection will be automatically shut down when it is done. In HTTP 1.1 mode, the server code has to manually set close_connection = True (or indicate the end of the response some other way).

Looking at the source code, changing protocol_version to "HTTP/1.1":

• Allows close_connection to be False by default (causing the problem above)
• Enables Expect: 100-continue handling (no problem)
• Changes the version actually used in the HTTP protocol (probably acceptable if everything else keeps working with a HTTP 1.1 client)
• Affects the SERVER_PROTOCOL variable for CGI scripts (unsure of the consequences)

Perhaps one way forward would be to use HTTP 1.1 by default, but still set close_connection = True. But then someone may come along and say Python should support persistent connections by default! It does not seem worth enabling HTTP 1.1 with non-persistent connections as the default.