New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TSNE workspaces exception #5977

Closed
AlexDBlack opened this Issue Jul 26, 2018 · 1 comment

Comments

Projects
None yet
1 participant
@AlexDBlack
Copy link
Member

AlexDBlack commented Jul 26, 2018

From @stoiandl on April 6, 2018 22:25

Issue Description

I am running a TSNE as per TSNEStandardExample.java but with the text file changed. It has the same format but it has 65,000 20-dim vectors. I am running 300 iterations.

When it reaches the second iteration I am getting the following exception regarding workspaces:

Exception in thread "main" org.nd4j.linalg.exception.ND4JIllegalStateException: Op [old_sub] X argument uses outdated workspace pointer from workspace [SPTREE_LOOP_EXTERNAL]

For more details, see the ND4J User Guide: nd4j.org/userguide#workspaces-panic

            at org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner.checkForWorkspaces(DefaultOpExecutioner.java:441)

            at org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner.profilingHookIn(DefaultOpExecutioner.java:490)

            at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:730)

            at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:119)

            at org.nd4j.linalg.api.ndarray.BaseNDArray.subi(BaseNDArray.java:3583)

            at org.nd4j.linalg.api.ndarray.BaseNDArray.sub(BaseNDArray.java:3264)

            at org.deeplearning4j.clustering.sptree.Cell.contains(Cell.java:62)

            at org.deeplearning4j.clustering.sptree.SpTree.insert(SpTree.java:171)

            at org.deeplearning4j.clustering.sptree.SpTree.insert(SpTree.java:202)

            at org.deeplearning4j.clustering.sptree.SpTree.fill(SpTree.java:397)

            at org.deeplearning4j.clustering.sptree.SpTree.<init>(SpTree.java:105)

            at org.deeplearning4j.clustering.sptree.SpTree.<init>(SpTree.java:117)

            at org.deeplearning4j.clustering.sptree.SpTree.<init>(SpTree.java:123)

            at org.deeplearning4j.plot.BarnesHutTsne.gradient(BarnesHutTsne.java:801)

            at org.deeplearning4j.plot.BarnesHutTsne.step(BarnesHutTsne.java:564)

            at org.deeplearning4j.plot.BarnesHutTsne.fit(BarnesHutTsne.java:534)

            at org.deeplearning4j.plot.BarnesHutTsne.fit(BarnesHutTsne.java:759)

Version Information

Please indicate relevant versions, including, if relevant:

  • Deeplearning4j version: 1.0.0-alpha
  • platform information (OS, etc): Windows 7 Enterprise, CPU
  • CUDA version, if used N/A
  • NVIDIA driver version, if in use N/A

Copied from original issue: deeplearning4j/nd4j#2821

AlexDBlack added a commit that referenced this issue Jul 26, 2018

AlexDBlack added a commit that referenced this issue Jul 27, 2018

Various ND4J Fixes (#5971)
* #5800 Clean up SameDiff inner interfaces/classes

* Fix Kuromoji resource loading path issue

* Fix WordVectorSerializerTest resource issue

* Another NLP iterator test fix

* Test file cleanup

* Fix Mmul shape calculation issue; un-ignore a bunch of now-passing samediff tests

* Fix issue with Eye op

* Un-ignore another set of now passing samediff op tests

* Fix result array zeroing issue with min/max/norm-max reductions

* More SameDiff fixes and test improvements

* Fix MeshGrid gradient

* Implement SequenceMask doDiff

* #5977 Fix TSNE/SpTree workspace issue (hopefully); add getters/setters

* Fix moments op backprop

* Fixes (+ test fixes) for index accumulation ops

* Partial fix for LogEntropy; other test fixes

* Allow rank 0 for SameDiff.var() methods; fix rank 0 import in TFGraphMapper

* FloorModOp import names fix
@lock

This comment has been minimized.

Copy link

lock bot commented Sep 21, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Sep 21, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.