You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a bug that has been present for a while in tsne-cuda and that we can't seem to track down. It appears to occur when the number of nodes allocated for the barnes-hut tree is exceeded by the tree building kernel. The way this is handled inside the code at the moment causes an infinite loop.
First of all - this should be impossible. Unless there are two points in exactly the same position, then it only takes 2N tree nodes to separate all the data perfectly. We've checked, and there aren't two points in exactly the same position.
Furthermore, increasing the number of allocated nodes only delays the problem, and doesn't solve it. Printing out the number of used nodes shows that this is below 2N nodes, and well below the new increased number of nodes, in the iteration before the infinite loop.
The bug also appears data/learning rate dependent. Some combinations of datasets, perplexities, and learning rates cause the bug, while others do not. It is at least partially deterministic because the same combination of dataset and input parameters will cause the bug in the same location. However, saving the state of the program (the current embedded positions, the input data, learning rate, etc.) and restarting it at that point does not seem to cause the bug.
The text was updated successfully, but these errors were encountered:
This is a bug that has been present for a while in tsne-cuda and that we can't seem to track down. It appears to occur when the number of nodes allocated for the barnes-hut tree is exceeded by the tree building kernel. The way this is handled inside the code at the moment causes an infinite loop.
First of all - this should be impossible. Unless there are two points in exactly the same position, then it only takes
2N
tree nodes to separate all the data perfectly. We've checked, and there aren't two points in exactly the same position.Furthermore, increasing the number of allocated nodes only delays the problem, and doesn't solve it. Printing out the number of used nodes shows that this is below
2N
nodes, and well below the new increased number of nodes, in the iteration before the infinite loop.The bug also appears data/learning rate dependent. Some combinations of datasets, perplexities, and learning rates cause the bug, while others do not. It is at least partially deterministic because the same combination of dataset and input parameters will cause the bug in the same location. However, saving the state of the program (the current embedded positions, the input data, learning rate, etc.) and restarting it at that point does not seem to cause the bug.
The text was updated successfully, but these errors were encountered: