doEvaluation should employ MemoryWorkspace for the output INDArray

[stolsvik]

I had problems running out of memory during evaluation. Turns out that a standard FF network explicitly does not use a workspace in doEvaluation when creating the output INDArray and then invoking evaluation.eval(..) - that is, the outputOfLayerDetached has the outputWorkspace set to null. This leads this the resulting INDArray to be "garbage-collector scoped", and I could by running watch on nvidia-smi see the memory building up (being consumed) while running through a quite large test dataset - and then getting a memory exception (since the GC didn't run during the entire loop).

By reimplementing the doEvaluation and invoking System.gc() in the loop, I avoided this, and getting zero buildup of memory - but obviously got a massive performance hit.

However, by instead reimplementing doEvaluation to employ a Memory WS, giving the ws to the outputOfLayerDetached (but still scopeOutOfWorkspaces(..) when invoking ROC.eval(..)), I got zero build-up of memory during that loop, and without any performance hit.

[AlexDBlack]

Yes, thanks for flagging, we definitely need to add this.
We might need "output in specified workspace" for RNN timestep too, for the split timeseries evaluation case.

[AlexDBlack]

Thanks for the issue - added here: #6295
Should be merged soon.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.