dev server doesn't read entire request body leading to client error

[rmorehead]

According to the HTTP spec, a server must read the entire request for HTTP 1.0 style request, or if the client supports "100 continue" via accept headers, the server must read entire request if the "100 continue" has been issued.

The issue above leads timing related CURLE_SEND_ERROR (55) with POSTS/PUTS via curl, and likely causes issues with other clients. The werkzeug server itself does not see this as an error as it has prematurely closed the socket on the client.

Since werkzeug does the socket closing and the 100 continue, it would be consistent if werkzeug ensured the entire request is read prior to closing the socket and not require the web application to add extra code/logic for every request.

To reproduce, have a werkzeug URL that accepts PUT or POST, but immediately responds with a redirect. Use curl to upload some data, 500kb should be plenty. Observe intermittent curl send errors.

[davidism]

The dev server does not drain input. Use a production server, this isn't a real issue during development.

[rmorehead]

If I submit an appropriate patch will you accept?

[davidism]

I can't say whether I'll accept a patch until I see it, but the help would definitely be appreciated.

[rmorehead]

Will do! I am embedding an httpbin server in a test script testing a CURL based embedded client, so having httpbin dev mode work smoothly is desirable. I have worked around for now.

[davidism]

Discussion from Flask: pallets/flask#2188

[davidism]

Closing. I don't want to add more stream handling to Werkzeug / the dev server. WSGI servers like Gunicorn can be used for input termination, and can be started programatically if you want to embed a production-level server in another application.

This should probably be addressed in Python itself, as we're just subclassing the built-in HTTP server.

[cipri-tom]

@davidism I understand your reluctance to add stream code to wekzeug, as it should never be used in production.

However, please consider that this is the default development server and this issue causes very subtle errors on the client, like it just did for me. I have been bitten by this before, but then I was testing with curl and could see the connection reset, which has directed me to the fix a bit sooner.

As this is the development server, I'd say it is worthwhile to have reasonable behaviour out of the box, since it is the first one that is used, especially by newcomers.

Thank you!

[mitsuhiko]

The WSGI spec is not clear who needs to handle it. My interpretation in Werkzeug was that this is the app's responsibility which would make this either a werkzeug or app bug, not server bug.

The original idea was that one would only respond when reading of form data is wanted but for error cases that's wrong. This means the draining code in Werkzeug's parser alone is insufficient.