-
-
Notifications
You must be signed in to change notification settings - Fork 31.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
http.server and SimpleHTTPServer hang after a few requests #75820
Comments
Doing a cross domain iframe test.
Loaded up Google chrome: 61.0.3163.100 (Official Build) (64-bit)
Same issue with python3
Note only one CTRL-C to exit... but note it didn't server domain3.html...so stuck in the same place as before. With python2 it serves domain3.html after hitting the first CTRL-C. With python3 it never serves it, just quits after the CTRL-C and browser is "spinning" waiting for the file.
I can reproduce this EVERY time, but I can't imagine everyone has this issue or it would have been fixed already, so tried to provide lots of info about my environment. Let me know if you need more. There seems to be a similar issue on windows: microsoft/WSL#1906 |
It has been pointed out to me that this issue may be related to chrome making multiple requests in parallel. A test with wget seems to support this. wget -E -H -k -K -p http://domain1.com:8000/domain1.html ...does not hang, whereas requests from chrome do hang. On some level I guess I wonder about the usefulness of simple web servers if they choke on very basic website requests from modern browsers. |
The change in handling KeyboardInterrupt was my intention in bpo-23430. I hope it isn’t a problem on its own :) Running the module with “python -m http.server” uses the HTTPServer class, based on socketserver.TCPServer. This only accepts one connection at a time, and waits for the SimpleHTTPRequestHandler class to finish handling each TCP connection before shutting it down and waiting for the next one. But SimpleHTTPRequestHandler supports persistent HTTP connections, meaning the TCP connection stays open after one HTTP request, ready for another. This prevents the simple TCPServer from accepting new connections. What is probably happening is the browser is trying to make one request (e.g. for domain3.html) on a new connection while it has an existing persistent connection already open. Having multiple host names pointing to the same server is not going to help; perhaps the browser does not realize that the two connections are to the same TCP server or that it could reuse the old connection. A simple workaround would be to use the socketserver.ThreadingMixIn or ForkingMixIn class. Each TCP connection will then be handled in a background thread. The mixed-in TCPServer will not wait for the handler, and will accept concurrent connections. If you want to avoid multiple threads, I think it is also possible to augment the server and handler classes to use “select” or similar so that the server will still handle each HTTP request one at a time, but can wait for requests on multiple TCP connections. But it requires subclassing and overriding some methods with custom code, and probably depends on deeper knowledge of how the classes work than is specified in the documentation. For existing versions of Python, I don’t there is much that could be done other than documenting the shortcomings of how a persistent HTTP connection vs multiple connections is handled. |
Actually take back a lot of what I wrote above. I forgot that SimpleHTTPRequestHandler only supports HTTP 1.0; I don’t think it uses keep-alive or persistent connections, so it should close its TCP connections promptly. There may be something else going on. Unfortunately I don’t have Chrome handy to experiment with. Perhaps it is holding a TCP connection open without making any request at all, and then trying to open a second connection. You would have to look at the TCP connections being created and shut down, and the HTTP requests being made, to verify. |
Same behavior on Windows. |
This probably has been around for a while: this 2011 thread in a Chromium wontfix bug is enlightening, but the solution suggested, a ThreadingMixIn for the HTTPServer, didn't help me. https://bugs.chromium.org/p/chromium/issues/detail?id=195550 |
I straced both chromium and Python during the issue and seen this: Chromium open a socket (port 55084), sends "GET /domain1.html" to it.
At this point we're stuck, three socket are opened, Python is reading on one of them, Chromium is writing on another. |
I wrote a 2 lines PR in which I propose to use threads to handle connections as it's well supported by socketserver via a ThreadingMixIn and fixes the issue. |
I don't think the PR as it stands is a good idea. These classes are designed to be composable, so it should be up to the library user whether or not to use threads. However it would be perfectly reasonable to choose to use threads in the 'test' function and thus the cli. Which fact should then be documented, and chromium can even be mentioned as one of the motivations for using threads in the cli server. |
I tried the approach in the PR "externally" (when starting the server using a test program), and I couldn't get it to work. But my test case was probably different: I was using Chrome rather than Chromium, and while they both work for me for simple HTTP file access to localhost without threading, I had tried to set up a PWA with a service worker, and maybe that does something different, so I got the hang, applied the Threading.mixIn, and it still hung. I don't know how to tell for sure if it is the same sort of hang, or something different, in the Windows environment. |
David you're right, I updated my patch, also added some documentation and a news entry. Glenn I can't tell if my PR fixes your issue without an strace or a test. It fixes the one I straced (pre-opening sockets leading to Python reading indefinitly on a socket which may never be used). If you could try my PR I would appreciate it. |
I don't know how to look back at the previous version of the PR, I was barely able to find the "current version" each time. The following line is in the current version: daemon_threads = True Whether it was in the previous version, I don't know, but I didn't notice it, but maybe I overlooked it due to other changes in the same area, which are now gone. This line was not in the old suggestion that I had found and tried. When I added it, my test case started working. I have no idea what the line really does, but the HTTP server is a daemon, and we are adding threading, so it sounds appropriate. I do wonder if it should somehow be put in the definition of ThreadedHTTPServer instead of "pass". And the old solution I had found had called the HTTPServer.__init__ which yours does not, which was surprising, but I'll not argue with success. |
Glenn you're right I modified my PR. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: