Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Some memory improvements #6883

Merged
merged 20 commits into from Dec 19, 2018

Conversation

@raver119
Copy link
Contributor

commented Dec 17, 2018

This PR adds:

  • INDArray & DataBuffer AutoCloseable (with couple of obvious restrictions)
    - GC handling improvements for CUDA backend
raver119 added 12 commits Dec 18, 2018
- varargs for few Nd4j.createFromArray signatures
- new method for memory tracking
- legacy tracking
- constant tracking

@raver119 raver119 merged commit 346f987 into master Dec 19, 2018

0 of 2 checks passed

codeclimate Code Climate encountered an error attempting to analyze this pull request.
Details
Codacy/PR Quality Review Not up to standards. This pull request quality could be better.
Details

@raver119 raver119 deleted the r119_close_that branch Dec 19, 2018

@@ -6488,7 +6488,7 @@ public void close() throws Exception {
return;

if (!closeable())
throw new ND4JIllegalStateException("Can't release view array");
throw new ND4JIllegalStateException("Can't release this INDArray");

This comment has been minimized.

Copy link
@stolsvik

stolsvik Dec 21, 2018

Would be great to know why it is not closeable.

for (val r:references)
r.markReleased();
}

@Override
public void close() {
if (!closeable())
throw new IllegalStateException("Can't release this data buffer");

This comment has been minimized.

Copy link
@stolsvik

stolsvik Dec 21, 2018

Would be great to know why it is not closeable.

printomi added a commit to printomi/deeplearning4j that referenced this pull request Jan 7, 2019
[WIP] Some memory improvements (eclipse#6883)
* initial commit

* not closeable for IsAttached()

* couple of simple tests

* one more test

* CUDA backend

* batched_gemm additional validation

* normalize_moments same mode fix

* - varargs for few Nd4j.createFromArray signatures
- new method for memory tracking

* one more test

* one more test

* workspace attached

* one more test

* - legacy tracking
- constant tracking

* cpu constant space

* minor tweaks

* important pico fix

* cpu workspace tracking

* separate tracking for workspace allocations

* temporary ignored test pack
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.