Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Ability to assign a specific workspace to the network / computation graph outputs #5932
When encountering system.gc() issues (nearly 100% CPU Time as we handle lots of objects simultaneously), we turned down periodic garbage collection as the documentation recommends :
Then, we understood we had to use Workspaces in order to manually handle INDArrays allocations/deallocations. But we encountered an issue with INDArrays created by output() or evaluate() calls of our computation graph.
Indeed those calls should be made outside of a Workspace otherwise we got :
Thus, the INDArrays returned were detached from any workspace and their memory would not be claimed until next system.gc() (which we try to avoid at most).
You suggested on gitter that it would be possible to give a computation graph output() or evaluate() calls a Workspace reference in order to be able to manage INDArrays native memory lifetime in an efficient way. That would indeed solve the above issue.
The global aim of this modification is to be able to serve a computation graph model with very high throughput without relying on system.gc() in order to free any allocated memory.
Thanks a lot for you kind answers.
Please indicate relevant versions, including, if relevant: