[Question] Inconsistent tensor size on SplitTable during backward propagation

[claudiogreco]

Hello,

I started to implement the model presented in Denil, Misha, et al. to model documents and sentences. I want to create embeddings for sentences and then apply some convolutional layers in order to extract relevant features. I started with this model:

require 'nn';

vocab = {hello=1, alex= 2, john=3, mary=4, lisa=5, home=6, school=7, goes=8}
vocab_size = 8

s1_d1 = {vocab['hello'], vocab['alex'], vocab['goes'], vocab['home']}
s2_d1 = {vocab['hello'], vocab['john'], vocab['goes'], vocab['school']}

s1_d2 = {vocab['hello'], vocab['lisa'], vocab['goes'], vocab['home']}
s2_d2 = {vocab['hello'], vocab['mary'], vocab['goes'], vocab['school']}

d1 = {s1_d1, s2_d1}
d2 = {s1_d2, s2_d2}

items = {d1, d2 }

embeddings_size = 10

document_model = nn.Sequential()
document_model:add(nn.Transpose({1, 2}))
document_model:add(nn.SplitTable(2))

sentences_model = nn.ParallelTable()

lookup_table = nn.LookupTable(vocab_size, embeddings_size)

sentence_model_1 = nn.Sequential()
sentence_model_1:add(lookup_table)
sentences_model:add(sentence_model_1)
-- some convolutions on sentence_model_1

sentence_model_2 = nn.Sequential()
sentence_model_2:add(lookup_table)
sentences_model:add(sentence_model_2)
-- some convolutions on sentence_model_2

document_model:add(sentences_model)
document_model:add(nn.JoinTable(1))
-- some convolutions on document_model

-- reshape to flatten output
document_model:add(nn.Reshape(80))

criterion = nn.MSECriterion()
pred = document_model:forward(torch.Tensor(d1))
print(pred)
output = torch.randn(80)
document_model:zeroGradParameters()
print(document_model:backward(input_doc, criterion:backward(document_model.output, output)))

I got the following error calling the backward method on document_model:

../lua/5.1/nn/SplitTable.lua:38: inconsistent tensor size ../torch/pkg/torch/lib/TH/generic/THTensorCopy.c:7

Apparently the problem is due to the LookupTable module that I have after a ParallelTable module because if I replace the lookup table with a linear layer it works fine.

How do you think I can solve it? Am I doing something wrong?

Thank you in advance for your help

[fmassa]

LookupTable doesn't have an updateGradInput method (it doesn't compute the gradients wrt the input), so the gradInput is not initialized (and by default it has 0 size), and that generates the error you are seeing.

I think one way to get around it is to replace the updateGradInput functions from all modules preceding the LookupTable to be empty functions. As you don't have parameters in the modules preceding the LookupTable, no need to compute gradients on it anyway.
In your example, it would consist of something like

-- preceeding code is the same
transpose = nn.Transpose({1, 2})
transpose.updateGradInput = function() end
sp = nn.SplitTable(2)
sp.updateGradInput = function() end

document_model = nn.Sequential()
document_model:add(transpose)
document_model:add(sp)
-- the rest is the same (but need to define input_doc to torch.Tensor(d1) !)

[claudiogreco]

Ok, it worked. In the while I improved my architecture because it was not right, too. Thank you for your help.