Processing a single graph in batch mode

[motiwari]

Hi @tkipf, thanks for your awesome work and providing this code!

I have a bit of a novice question: I'm trying to process my graph's features through a GCN in mini-batches. I.e. let's say I have a 1000-node graph and I want to process it through the GCN in mini-batches of size 50.

It doesn't seem like the code currently supports this because we have to multiply by the full adjacency matrix in the GCN layer's forward pass - do you have any sense of how I can support these "batch" operations? Better yet, do you have example code that does this?

My code looks something like:

z0 = F.tanh(self.gcn1(x, self.fancy_adj))

where x is a sampled batch of size 50 x F (not the full batch of 1000 x F) and self.fancy_adj is the adjacency matrix transformed as suggested in your paper ( adjacency + identity + row normalized). The problem, of course, is that self.fancy_adj is a 1000 x 1000 matrix. Even if I take just the rows of self.fancy_adj corresponding to the 50 points in the batch, then the adjacency matrix becomes a 50 x 1000 matrix which can't be multiplied by the 50 x F sampled batch.

[tkipf]

Great question! This has come up in my other repository on GCNs and I’ve outlined a solution here: tkipf/gcn#4

…

On Sun 14. Oct 2018 at 20:50 motiwari ***@***.***> wrote: Hi @tkipf <https://github.com/tkipf>, thanks for your awesome work and providing this code! I have a bit of a novice question: I'm trying to process my graph's features through a GCN in mini-batches. I.e. let's say I have a 1000-node graph and I want to process it through the GCN in mini-batches of size 50. It doesn't seem like the code currently supports this because we have to multiply by the full adjacency matrix in the GCN layer's forward pass - do you have any sense of how I can support these "batch" operations? Better yet, do you have example code that does this? My code looks something like: z0 = F.tanh(self.gcn1(x, self.fancy_adj)) where x is a *sampled* batch of size 50 x F (not the full batch of 1000 x F) and self.fancy_adj is the adjacency matrix transformed as suggested in your paper ( adjacency + identity + row normalized) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#23>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHAcYI2Fcr3J4q1m0pIk-x2exyksbkHbks5uk5VpgaJpZM4XbTEj> .

[motiwari]

Thanks @tkipf ! I saw that thread but it looks like it only discusses processing multiple graphs as a block-diagonal matrix, but I'm asking about a single large graph whose adjacency matrix is too large to fit on a single device. Am I missing something?

[tkipf]

In this case, have a look at FastGCN: https://arxiv.org/abs/1801.10247