Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TSNE workspaces exception #5977

Closed
AlexDBlack opened this issue Jul 26, 2018 · 1 comment · Fixed by #5971

Comments

@AlexDBlack
Copy link
Contributor

commented Jul 26, 2018

From @stoiandl on April 6, 2018 22:25

Issue Description

I am running a TSNE as per TSNEStandardExample.java but with the text file changed. It has the same format but it has 65,000 20-dim vectors. I am running 300 iterations.

When it reaches the second iteration I am getting the following exception regarding workspaces:

Exception in thread "main" org.nd4j.linalg.exception.ND4JIllegalStateException: Op [old_sub] X argument uses outdated workspace pointer from workspace [SPTREE_LOOP_EXTERNAL]

For more details, see the ND4J User Guide: nd4j.org/userguide#workspaces-panic

            at org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner.checkForWorkspaces(DefaultOpExecutioner.java:441)

            at org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner.profilingHookIn(DefaultOpExecutioner.java:490)

            at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:730)

            at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:119)

            at org.nd4j.linalg.api.ndarray.BaseNDArray.subi(BaseNDArray.java:3583)

            at org.nd4j.linalg.api.ndarray.BaseNDArray.sub(BaseNDArray.java:3264)

            at org.deeplearning4j.clustering.sptree.Cell.contains(Cell.java:62)

            at org.deeplearning4j.clustering.sptree.SpTree.insert(SpTree.java:171)

            at org.deeplearning4j.clustering.sptree.SpTree.insert(SpTree.java:202)

            at org.deeplearning4j.clustering.sptree.SpTree.fill(SpTree.java:397)

            at org.deeplearning4j.clustering.sptree.SpTree.<init>(SpTree.java:105)

            at org.deeplearning4j.clustering.sptree.SpTree.<init>(SpTree.java:117)

            at org.deeplearning4j.clustering.sptree.SpTree.<init>(SpTree.java:123)

            at org.deeplearning4j.plot.BarnesHutTsne.gradient(BarnesHutTsne.java:801)

            at org.deeplearning4j.plot.BarnesHutTsne.step(BarnesHutTsne.java:564)

            at org.deeplearning4j.plot.BarnesHutTsne.fit(BarnesHutTsne.java:534)

            at org.deeplearning4j.plot.BarnesHutTsne.fit(BarnesHutTsne.java:759)

Version Information

Please indicate relevant versions, including, if relevant:

  • Deeplearning4j version: 1.0.0-alpha
  • platform information (OS, etc): Windows 7 Enterprise, CPU
  • CUDA version, if used N/A
  • NVIDIA driver version, if in use N/A

Copied from original issue: deeplearning4j/nd4j#2821

@AlexDBlack AlexDBlack added this to the DL4J/Arbiter/DataVec Next Steps milestone Jul 26, 2018

AlexDBlack added a commit that referenced this issue Jul 26, 2018
AlexDBlack added a commit that referenced this issue Jul 27, 2018
Various ND4J Fixes (#5971)
* #5800 Clean up SameDiff inner interfaces/classes

* Fix Kuromoji resource loading path issue

* Fix WordVectorSerializerTest resource issue

* Another NLP iterator test fix

* Test file cleanup

* Fix Mmul shape calculation issue; un-ignore a bunch of now-passing samediff tests

* Fix issue with Eye op

* Un-ignore another set of now passing samediff op tests

* Fix result array zeroing issue with min/max/norm-max reductions

* More SameDiff fixes and test improvements

* Fix MeshGrid gradient

* Implement SequenceMask doDiff

* #5977 Fix TSNE/SpTree workspace issue (hopefully); add getters/setters

* Fix moments op backprop

* Fixes (+ test fixes) for index accumulation ops

* Partial fix for LogEntropy; other test fixes

* Allow rank 0 for SameDiff.var() methods; fix rank 0 import in TFGraphMapper

* FloorModOp import names fix
@lock

This comment has been minimized.

Copy link

commented Sep 21, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Sep 21, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
1 participant
You can’t perform that action at this time.