Strange Leak on HTTPConnect/IOStream #760

Closed
wsantos opened this Issue Apr 26, 2013 · 2 comments

Projects

None yet

2 participants

@wsantos
Contributor
wsantos commented Apr 26, 2013

I don't get why, the first HTTPConnection will never be collected ex.: objgraph

The IOStream from the left will never be collected from GC, any glue ? i can't see any reason for wrapper/cell poit to the IOStream.

@bdarnell
Member

objgraph tends to cut things off a little too soon by default. Try passing max_depth=5 or 10 to show_backrefs.

@wsantos
Contributor
wsantos commented Apr 28, 2013

i think the "problem" is here

    def _finish_request(self):
        print("fim request")
        if self.no_keep_alive:
            disconnect = True
        else:
            connection_header = self._request.headers.get("Connection")
            if connection_header is not None:
                connection_header = connection_header.lower()
            if self._request.supports_http_1_1():
                disconnect = connection_header == "close"
            elif ("Content-Length" in self._request.headers
                    or self._request.method in ("HEAD", "GET")):
                disconnect = connection_header != "keep-alive"
            else:
                disconnect = True
        self._request = None
        self._request_finished = False
        self._clear_callbacks()
        if disconnect:
            self.close()
            return
        try:
            # Use a try/except instead of checking stream.closed()
            # directly, because in some cases the stream doesn't discover
            # that it's closed until you try to read from it.

            self.stream.read_until(b"\r\n\r\n", self._header_callback)
        except iostream.StreamClosedError:
            self.close()

We keep setting, _header_callback to make usage of a persistent connection, but if we never receive the close ? when the stream/connection is closed ?

@bdarnell bdarnell added a commit that closed this issue Apr 29, 2013
@bdarnell bdarnell Fix a memory leak in HTTPServer with very large file uploads.
This is not quite a true leak since the garbage collector will
reclaim everything when it runs, but it might as well be a leak
because CPython's garbage collector uses heuristics based on the
number of allocated objects to decide when to run.  Since this
scenario resulted in a small number of very large objects, the process
could consume a large amount of memory.

The actual change is to ensure that HTTPConnection always sets a
close_callback on the IOStream instead of waiting for the Application
to set one.  I also nulled out a few more references to break up cycles.

Closes #740.
Closes #747.
Closes #760.
12780c1
@bdarnell bdarnell closed this in 12780c1 Apr 29, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment