-
Notifications
You must be signed in to change notification settings - Fork 217
SavedModelBundle leak fix #335
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Turns out |
@@ -510,21 +508,18 @@ private static SavedModelBundle load( | |||
TF_Graph graph = TF_NewGraph(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We probably should call TF_Graph.newGraph()
here, but I'm not sure...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried that initially, but Graph.close()
doesn't use handle.releaseReference()
like Session.close()
does, and so it doesn't call the TF_Graph
deallocator. I think you're the last person to touch both Graph.delete
and Session.delete
and I wasn't sure if they were different for a reason.
As I said in the comment above, I think they should be made consistent, but that's going to be a bigger change than just the one which fixes this leak. For example if we switch Graph
over to use handle.releaseReference()
and then use TF_Graph.newGraph
everywhere it'll mean that the TF_Graph
inside Graph
is now sensitive to enclosing PointerScope
s which it currently isn't.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it that the TF_Graph becomes managed by the TF_Session and we shouldn't touch it anymore after that? It sounds like that from the documentation of the C API. In that case I'd put the call to TF_NewGraph() in TF_Session.loadSessionFromSavedModel() and forget about it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I think the graph is separate from the session, or at least that's my reading of the doc here - https://github.com/tensorflow/tensorflow/blob/master/tensorflow/c/c_api.h#L1229. The TF_NewSession
doc above it suggests the TF_Graph
is kept alive by the TF_Session
, but it doesn't say that the session owns the graph, so I think we're still responsible for closing it.
@Craigacp Would definitely a +1 to say that backporting this fix to a |
This PR adds the deallocators to the
TF_Graph
andTF_Session
that live inside aSavedModelBundle
as previously they didn't free their memory onSavedModelBundle.close()
.We may want to backport this to the 0.3.x branch. I think the main conflict there is that there are formatting changes in
master
not present inr0.3
but we could just write a similar patch, it's only 5 lines.cc @saudet