HTTP Patch raising BrokenPipeError when 100 Continue present? #2503
-
|
Hey, I am trying to properly handle 413 return code, but seems like httpx doesn't like it. The response request: When making this request via httpx self._http = httpx.AsyncClient(
auth=(_config.hub_access_key_id, _config.hub_access_key_secret),
verify=False,
)
# ...
res = await self._http.patch(url, content=patch)I get Interesting thing though is that when using native res = requests.patch(url, data=patch, auth=(_config.hub_access_key_id, _config.hub_access_key_secret))Any ideas? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 9 replies
-
|
The traceback shows the server closing the connection before the request body has been fully sent. Which is reasonable behaviour, but curious that you're not seeing the same behaviour with
|
Beta Was this translation helpful? Give feedback.
-
My expectation would be that In contrast, I would be interested in a closer comparison of the behaviour of |
Beta Was this translation helpful? Give feedback.
-
|
Min repro: Testing payload package main
import "net/http"
type userHandler struct{}
func (h *userHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
switch {
case r.Method == http.MethodPatch:
http.Error(w, http.StatusText(http.StatusRequestEntityTooLarge), http.StatusRequestEntityTooLarge)
return
default:
http.Error(w, http.StatusText(http.StatusNotFound), http.StatusNotFound)
return
}
}
func main() {
mux := http.NewServeMux()
mux.Handle("/", &userHandler{})
http.ListenAndServe(":8080", mux)
}Requires golang installed, tested against # client.py
import httpx
import requests
# Read 10mb.csv
with open('10mb.csv', 'r') as f:
data = f.read()
res = requests.patch("http://localhost:8080", data=data)
# res = httpx.patch("http://localhost:8080", data=data)
print(res.status_code)curl output curl -i -X PATCH -d @10mb.csv http://127.0.0.1:8080
HTTP/1.1 413 Request Entity Too Large
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Tue, 13 Dec 2022 12:20:16 GMT
Content-Length: 25
Connection: close
Request Entity Too LargeIf interested, here is a python server repro attempt # server.py
import http.server
import socketserver
PORT = 8080
class MyHandler(http.server.SimpleHTTPRequestHandler):
def do_PATCH(self):
# No error is thrown if payload is ~10mb and the body is read. Why?
# content_length = int(self.headers['Content-Length'])
# body = self.rfile.reath)
self.send_response(code=413, message="Request Entity Too Large")
self.send_header("Connection", "close")
self.send_header("X-Content-Type-Options", "nosniff")
self.send_header("Content-Type", "text/plain; charset=utf-8")
self.end_headers()
try:
with socketserver.TCPServer(("", PORT), MyHandler) as httpd:
print(f"Starting http://0.0.0.0:{PORT}")
httpd.serve_forever()
except KeyboardInterrupt:
print("Stopping by Ctrl+C")
httpd.server_close() # to resolve problem `OSError: [Errno 98] Address already in use` |
Beta Was this translation helpful? Give feedback.
-
|
Okay, thanks for the comprehensive report. This isn't related to "100 continue", but is about the case where the server half-closes while reading the request, but still sends a valid response. The relevant part of I can observe that if we take the existing "write the request"... with Trace("http11.send_request_headers", request, kwargs) as trace:
self._send_request_headers(**kwargs)
with Trace("http11.send_request_body", request, kwargs) as trace:
self._send_request_body(**kwargs)And instead catch and ignore the case where we get a try:
with Trace("http11.send_request_headers", request, kwargs) as trace:
self._send_request_headers(**kwargs)
with Trace("http11.send_request_body", request, kwargs) as trace:
self._send_request_body(**kwargs)
except WriteError as exc:
passThen we gracefully fall through to the "read response" part and the 413 response is received okay. I can see that We'll want to escalate this into an issue, and get it resolved in |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for a prompt resolution @tomchristie!
Do you want me to file the issue or do you want to do it yourself?
What does half-closing mean in this context? |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
The traceback shows the server closing the connection before the request body has been fully sent. Which is reasonable behaviour, but curious that you're not seeing the same behaviour with
requests.httpxrequest with a syncrequestsrequest is a better starting point.