New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory Leak #606

Closed
leetreveil opened this Issue Aug 11, 2018 · 4 comments

Comments

Projects
None yet
2 participants
@leetreveil

leetreveil commented Aug 11, 2018

There's a memory leak in APIStar that is resulting in the allocation of around 3kB (YMMV) of mem per second that is not being collected by the GC.

mem-graph-leak

Repro:

from apistar import App, Route, http

def welcome():
    return {'message': 'Welcome to API Star!'}

class LeakingHook:
    def on_request(self, req: http.Request):
        pass

routes = [
    Route('/', method='GET', handler=welcome)
]

app = App(routes=routes, event_hooks=[LeakingHook])


if __name__ == '__main__':
    app.serve('127.0.0.1', 5000)

To reproduce this start the above script and hit the welcome endpoint with apache bench:

while; do ab -n 100000 -c 100 127.0.0.1:5000/; done

I believe the memory leak is coming from the inspect call here:

return_annotation = inspect.signature(self.resolve).return_annotation

More on that soon once I instrument the app with tracemalloc.

@leetreveil

This comment has been minimized.

leetreveil commented Aug 13, 2018

And here's the results from tracemalloc:

[ Top 10 differences ]
/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/inspect.py:2760: size=1802 KiB (+1802 KiB), count=10979 (+10979), average=168 B
/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/inspect.py:2724: size=1801 KiB (+1801 KiB), count=10978 (+10978), average=168 B
/usr/local/lib/python3.6/site-packages/apistar/server/injector.py:81: size=1175 KiB (+1175 KiB), count=12536 (+12536), average=96 B
<frozen importlib._bootstrap_external>:487: size=1166 KiB (+1166 KiB), count=12589 (+12589), average=95 B
/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/inspect.py:510: size=772 KiB (+772 KiB), count=10985 (+10985), average=72 B
/usr/local/lib/python3.6/site-packages/apistar/server/injector.py:30: size=772 KiB (+772 KiB), count=10977 (+10977), average=72 B
/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/linecache.py:137: size=502 KiB (+502 KiB), count=4992 (+4992), average=103 B
/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/inspect.py:3037: size=490 KiB (+490 KiB), count=7836 (+7836), average=64 B
/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tracemalloc.py:449: size=442 KiB (+442 KiB), count=4399 (+4399), average=103 B
/usr/local/lib/python3.6/site-packages/apistar/server/components.py:14: size=416 KiB (+416 KiB), count=7840 (+7840), average=54 B
127.0.0.1 - - [13/Aug/2018 14:43:54] "GET / HTTP/1.0" 200 -

And a full traceback from one of the inspect.signature calls that's leaking:

328 memory blocks: 53.8 KiB
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/inspect.py", line 2760
    for param in parameters))
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/inspect.py", line 2173
    __validate_parameters__=is_duck_function)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/inspect.py", line 2262
    return _signature_from_function(sigcls, obj)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/inspect.py", line 2195
    sigcls=sigcls)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/inspect.py", line 2787
    follow_wrapper_chains=follow_wrapped)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/inspect.py", line 3037
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
  File "/usr/local/lib/python3.6/site-packages/apistar/server/injector.py", line 32
    signature = inspect.signature(func)
  File "/usr/local/lib/python3.6/site-packages/apistar/server/injector.py", line 69
    parent_parameter=parameter
  File "/usr/local/lib/python3.6/site-packages/apistar/server/injector.py", line 69
    parent_parameter=parameter
  File "/usr/local/lib/python3.6/site-packages/apistar/server/injector.py", line 89
    func_steps = self.resolve_function(func, seen_state=seen_state, set_return=True)
  File "/usr/local/lib/python3.6/site-packages/apistar/server/injector.py", line 100
    steps = self.resolve_functions(funcs)
  File "/usr/local/lib/python3.6/site-packages/apistar/server/app.py", line 227
    return self.injector.run(funcs, state)
  File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 258
    application_iter = app(environ, start_response)
  File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 270
    execute(self.server.app)
  File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 328
    return self.run_wsgi()
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/server.py", line 418
    self.handle_one_request()
  File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 293
    rv = BaseHTTPRequestHandler.handle(self)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socketserver.py", line 696
    self.handle()
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socketserver.py", line 361
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socketserver.py", line 348
    self.finish_request(request, client_address)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socketserver.py", line 317
    self.process_request(request, client_address)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socketserver.py", line 238
    self._handle_request_noblock()
  File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 612
    HTTPServer.serve_forever(self)
  File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 777
    srv.serve_forever()
  File "/usr/local/lib/python3.6/site-packages/werkzeug/serving.py", line 814
    inner()

Repro scripts: https://gist.github.com/leetreveil/f36e3ff998e64cce44ee0bb21b7cbcce

@leetreveil

This comment has been minimized.

leetreveil commented Aug 13, 2018

I can also confirm that using old-style event hooks:

# Old style usage, to be deprecated on the next version bump.

Do not leak. So it's highly likely that the introduction of new style event hooks that get instantiated on every request are the cause:

https://github.com/encode/apistar/pull/475/files

@leetreveil

This comment has been minimized.

leetreveil commented Aug 13, 2018

Found it with the help of objgraph.

chain

New style event hooks get put into the resolver cache on every request:

steps = self.resolver_cache[funcs]

@tomchristie

This comment has been minimized.

Member

tomchristie commented Sep 25, 2018

Closing this off given that 0.6 is moving to a framework-agnostic suite of API tools, and will no longer include the server. See https://discuss.apistar.org/t/api-star-as-a-framework-independant-tool/614 and #624.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment