New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to assign a specific workspace to the network / computation graph outputs #5932

Closed
NicolasVidal opened this Issue Jul 19, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@NicolasVidal
Copy link

NicolasVidal commented Jul 19, 2018

Issue Description

When encountering system.gc() issues (nearly 100% CPU Time as we handle lots of objects simultaneously), we turned down periodic garbage collection as the documentation recommends :
(https://deeplearning4j.org/workspaces#garbage-collector)

Then, we understood we had to use Workspaces in order to manually handle INDArrays allocations/deallocations. But we encountered an issue with INDArrays created by output() or evaluate() calls of our computation graph.

Indeed those calls should be made outside of a Workspace otherwise we got :

Exception in thread "main" org.nd4j.linalg.workspace.ND4JWorkspaceException: Expected no workspace active before call to outputOfLayersDetached - Open/active workspaces: [OOMPUTE_OUTPUT_WORKSPACE]

Thus, the INDArrays returned were detached from any workspace and their memory would not be claimed until next system.gc() (which we try to avoid at most).

You suggested on gitter that it would be possible to give a computation graph output() or evaluate() calls a Workspace reference in order to be able to manage INDArrays native memory lifetime in an efficient way. That would indeed solve the above issue.

The global aim of this modification is to be able to serve a computation graph model with very high throughput without relying on system.gc() in order to free any allocated memory.

Thanks a lot for you kind answers.

Version Information

Please indicate relevant versions, including, if relevant:

  • Deeplearning4j version : 1.0.0-beta
  • platform information (OS, etc) : Windows 10
  • CUDA version, if used : 9.0 and 9.1
  • NVIDIA driver version, if in use : 398.36
@AlexDBlack

This comment has been minimized.

Copy link
Member

AlexDBlack commented Jul 25, 2018

Implemented here, will be merged soon: #5962

AlexDBlack added a commit that referenced this issue Jul 25, 2018

DL4J: MLN/CG output overloads with output arrays in specified workspa…
…ces (#5962)

* #5932 Output in specified workspace - MultiLayerNetwork

* #5932 Output in specified workspace - ComputationGraph
@lock

This comment has been minimized.

Copy link

lock bot commented Sep 21, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Sep 21, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.