Number nodes at each layers

[tamthuc1995]

In the unsupervised model, I notice that it use the second layers's num_samples as the number of neighbor nodes for the first layers. This lead to samples shape for next aggregate function become this:

• batch_size x [1, 10, 25*10] (which is different from the paper that first layers 25 neighbor nodes, second layer is 10 neighbor nodes)

These line:

• https://github.com/williamleif/GraphSAGE/blob/master/graphsage/models.py#L268-L275

• https://github.com/williamleif/GraphSAGE/blob/master/graphsage/models.py#L299

• https://github.com/williamleif/GraphSAGE/blob/master/graphsage/models.py#L326-L327

Thank you, I learned a lot from the code

[bkj]

If I understand your question correctly, the way it works is:

• sample a "center" node, n0
• sample 10 1-hop neighbors, n1
• sample 25 1-hop neighbors of each n1, yielding 250 2-hop neighbors of n0

Does that make sense?

[tamthuc1995]

yes, and it's seem to me that it is different from what I read in the paper.
In addition, I also want to ask about those parameter's effects to the model when the max_degree is ~ 1000-2000.
Last thing, do you think using other neighbors selection method will improving the model ? Since not all nodes have the same contribution to its neighbors, maybe "adding features of edges".

[williamleif]

Hi,

Yes the sampling can be a bit counter-intuitive and we tried to clear this up in the newest arxiv version. The description by @bkj is correct. Hopefully the new arxiv minibatch description is more clear, as it directly matches @bkj's description. Apologies for the confusion!

In general, we (and other researchers) have found that increasing the neighborhood sample size improves performance but that the performance quickly plateaus by around 25-50 neighbors. This general trend holds for a number of datasets and has been replicated in a couple ICLR submissions.

Of course, in a network with extremely high average degree or atypical degree distribution things might be different, but right now this is an empirical question and so far, we have found that the model can achieve near optimal performance with <50 sampled neighbors across all datasets tested. In future work, we hope to characterize this tradeoff more formally with regards to graph properties, such as density and clustering.

[tamthuc1995]

Thank you for your answers. I'm really fascinated by your work and hope to see the new arxiv version.