New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Custom losses, coherent embeddings #35
Comments
It is a little hard to do that and still maintain efficiency; potentially it could be added as an extra code-path on the side that is slower but more flexible. That would be a more significant project however. |
Thank you for the prompt reply! def tsne_kl_loss(points, P):
n = tf.shape(points)[0]
Q = 1.0 / (1.0 + pdist2(points))
sQ = tf.reduce_sum(Q) - tf.cast(n, tf.float32)
return tf.reduce_sum(P*tf.log(P/Q)) + tf.log(sQ) Then one can combine this loss with others and use one of standard optimizers. |
At that level, yes almost certainly; if you are willing to do N^2 work then you can certainly have custom loss -- I was generally seeking to avoid that. On that front you might be interested by smallvis which implements t-SNE, LargeVis and UMAP in a common framework which I suspect would easily be adaptable to custom loss functions. The cathc is that it only supports small datasets for exactly the reason cited above. As a way to experiment, however, it is quite powerful. |
Thank you for pointing me to smallvis! I think that's exactly what I needed. Still, I like to think of algorithmic optimizations, like employing BH or something else as yet another, partially orthogonal component. |
How to obtain the matrix P from UMAP? Is it self.graph_ from fuzzy_simplicial_set function? Thanks! |
That is the equivalent of it, yes. |
Nice property of TSNE, that is not exploited in most of implementations, is that it can be treated as a combination of two orthogonal components: loss function and optimization algorithm. For example one may visualize set on temporally varying vectors with a sequence on coherent embeddings by adding a loss term that penalizes unnecessary movement of each vector between those embeddings. Is it possible to have provide such flexibility to use additional constrains with UMAP?
The text was updated successfully, but these errors were encountered: