Continuous Space

[bc]

Hi~

I enjoyed reading your paper this morning. I am curious whether the techniques you mention here could be extended to networks which yield a real-number output?
Thanks,
Brian

[tianlema]

Hi Brian,
Thank you for your interest!
Yes, this technique can be extended to networks that yield continuous numbers.

In this paper, we mainly developed a kNN attention pooling layer (an extension to Graph Attention Model (GAM), https://arxiv.org/abs/1710.10903), which is just like a convolutional layer and can be stacked together to learn a network embedding (i.e., finding a good representation for each node in the network).

If some class labels are known (classification task), then a typical model architecture is like this.

The output of the stacked kNN attention pooling layer is in fact a network embedding. It's similar to the last layer before fully connected layer in CNN for computer vision related tasks.

With the newly learned representation, we can add a classification head if some class labels are known.
And for continuous output (regression task), we can just add a regression head, and use loss function for regression (e.g., MSE loss).

Besides, our model can handle not just network data, but in general, any set of objects. Given a set of objects, for example, a number of patients with the same type of cancer but different cancer subtypes, we usually don't know a network among them a priori. But we can calculate an affinity (similarity) network using their latent representations. We use this affinity network to select k-nearest neighbors and make them have similar representations. And then calculate new affinity network, repeat again.

Eventually we learned an embedding (vector representations) for each node in the "network". This new network embedding can used for downstream classification or clustering tasks.

If you have any further questions, feel free to contact me.