Question about sampling methods in minibatch

[cxfneo]

Hi

I am reading your code( now only mean aggregate). I am confused why the sampling method is in a reversed layer order?

def sample(self, inputs, layer_infos, batch_size=None):
""" Sample neighbors to be the supportive fields for multi-layer convolutions.

    Args:
        inputs: batch inputs
        batch_size: the number of inputs (different for batch inputs and negative samples).
    """
    
    if batch_size is None:
        batch_size = self.batch_size
    samples = [inputs]
    # size of convolution support at each layer per node
    support_size = 1
    support_sizes = [support_size]
    for k in range(len(layer_infos)):
        t = len(layer_infos) - k - 1
        support_size *= layer_infos[t].num_samples
        sampler = layer_infos[t].neigh_sampler
        node = sampler((samples[k], layer_infos[t].num_samples))
        samples.append(tf.reshape(node, [support_size * batch_size,]))
        support_sizes.append(support_size)
    return samples, support_sizes

[williamleif]

Hi,

Yes this can be confusing/counterintuitive and has to do with the naming conventions we use, as well how we "unfold" the recursion/loop of Algorithm 1 in our paper. We are making considerable efforts to clarify this in the next version of our paper, as well in updates to the code.

The short answer is that this is just a convention, which is complicated by the fact that the minibatch sampling is conceptually reversed compared to the outer loop (over "k") in Algorithm 1 (in our paper). By convention, when we refer to "layer k" in the code, we (usually) mean the computations/nodes present at "depth k" in Algorithm 1. But the key thing to note is that the sampling in our minibatch algorithm is in reverse order compared to outer loop (iteration over k) in Algorithm 1.

In particular, say we want to generate embeddings for a set of input, or target, nodes in a minibatch with depth K=2 and with neighborhood sample sizes S_1=10 and S_2=25 (i.e., we are using information from 2-hop neighborhoods), then, in our current convention, "layer 0" nodes (corresponding to the k=0 base case in Algorithm 1) are the nodes that are 2-hops away from the target nodes (and we need 10*25 of them per input node), "layer-1" nodes are 1-hop away (and we need 10 of them per input node), and "layer-2" nodes are target/input nodes.

Put in another way, the representations we want to generate -- for the input/target nodes -- are at depth/layer k=2 and these representations recursively depend on representations of the input nodes's neighbors at the previous layers. So from the perspective of sampling, we must sample the higher-depth/layer nodes first.

Anyways, I hope this made some sense, as this was a somewhat rushed reply. In any event, we will aim to clarify this in much more detail in updates to come.

[williamleif]

Closing for lack of activity. Feel free to email me if you have more conceptual questions about the algorithm.

[cxfneo]

thank u for your reply. Basically understand it.

…

-- 发自我的网易邮箱手机智能版 在 2017-10-10 03:42:34，"William L Hamilton" <notifications@github.com> 写道： Hi, Yes this can be confusing/counterintuitive and has to do with the naming conventions we use, as well how we "unfold" the recursion/loop of Algorithm 1 in our paper. We are making considerable efforts to clarify this in the next version of our paper, as well in updates to the code. The short answer is that this is just a convention, which is complicated by the fact that the minibatch sampling is conceptually reversed compared to the outer loop (over "k") in Algorithm 1 (in our paper). By convention, when we refer to "layer k" in the code, we (usually) mean the computations/nodes present at "depth k" in Algorithm 1. But the key thing to note is that the sampling in our minibatch algorithm is in reverse order compared to outer loop (iteration over k) in Algorithm 1. In particular, say we want to generate embeddings for a set of input, or target, nodes in a minibatch with depth K=2 and with neighborhood sample sizes S_1=10 and S_2=25 (i.e., we are using information from 2-hop neighborhoods), then, in our current convention, "layer 0" nodes (corresponding to the k=0 base case in Algorithm 1) are the nodes that are 2-hops away from the target nodes (and we need 10*25 of them per input node), "layer-1" nodes are 1-hop away (and we need 10 of them per input node), and "layer-2" nodes are target/input nodes. Put in another way, the representations we want to generate -- for the input/target nodes -- are at depth/layer k=2 and these representations recursively depend on representations of the input nodes's neighbors at the previous layers. So from the perspective of sampling, we must sample the higher-depth/layer nodes first. Anyways, I hope this made some sense, as this was a somewhat rushed reply. In any event, we will aim to clarify this in much more detail in updates to come. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.