Sort sequences internally in pack_padded_sequence

[colesbury]

Currently, pack_padded_sequence requires the caller to sort inputs by the length (descending). We should sort and reorder the input internally if it's not already sorted. We can then restore the original order in pad_packed_sequence.

cc @kyunghyuncho

[hhsecond]

I am working on 1128. I can add this to my code as well

[apaszke]

This will make it impossible to do the batched filling we do now

[colesbury]

@apaszke, you can sort and then do the batched filling

[apaszke]

but how do you sort a tensor? with a series of selects and cat? it's expensive

[colesbury]

A single call to index_select

…

On Thu, Nov 9, 2017 at 2:23 PM Adam Paszke ***@***.***> wrote: but how do you sort a tensor? with a series of selects and cat? it's expensive — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3584 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAoB-hCFQG8UvJVnuJfTVdP1LmS3LTnvks5s01FJgaJpZM4QXUD4> .

[sanyam5]

Would really love to have this feature! Has this been implemented yet?

[elanmart]

But you'd still have to manually sort all other tensors in a batch, no?

[sanyam5]

Well, I was hoping for a method that would first sort the inputs, then compute the results and then restore the original order before returning the results. This way everything would happen under-the-hood and the user need not worry about sorting anything else in their code.

[junpeiz]

If we manually do the sort before RNN and unsort after the RNN, would it influence the backpropagation? For example,

def forward(self, input, lengths, hidden):
    # Sort the input and lengths as the descending order
    lengths, perm_index = lengths.sort(0, descending=True)
    input = input[perm_index]

    packed_input = pack(input, list(lengths.data), batch_first=True)
    output, hidden = self.rnn(packed_input, hidden)
    output = unpack(output, batch_first=True)[0]
    
    # restore the sorting
    odx = perm_index.view(-1, 1).unsqueeze(1).expand(output.size(0), output.size(1), output.size(2))
    decoded = output.gather(0, odx)
    return decoded, hidden

[RadZaeem]

@zhoujunpei
in my experience, I managed to do backprop by also sorting the labels with the same ordering as the lengths list.
labels = labels[perm_index]
loss = CrossEntropy(out, labels)
loss.backward()

if not, the backprop will fit into garbage (random labels)

maybe your method also works. the point is the ouput and label index is same

[cdluminate]

second this proposal

[airsplay]

For efficiency, is it possible to check whether the sequence is sorted inside the function "pack_padded_sequence"?
It would be much convenient if the descending requirement is not mandatory.

[cdluminate]

Excellent!