I cannot figure out the inputs to MIPModel

[lixiangchun]

Hope anyone could help figure out the inputs to MIPModel.

import mip_model as mip
image_model = mip.Densenet()
net = mip.MIPModel(image_model, 1024, 64, 2, 2, 1024, 0.1, 0.1, 'sum')
B1, N1, C1, H1, W1 = 2, 4, 3, 224, 224
images = torch.rand(B1, N1, C1, H1, W1)
x = net(images, torch.tensor([-30, -20, -10, 0]), torch.tensor([N1]))
print(x.shape)

Errors:

Traceback (most recent call last):
  File "mip_model.py", line 305, in <module>
    net(images, torch.tensor([-30, -20, -10, 0]), torch.tensor([N1]))
  File "/opt/software/install/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "mip_model.py", line 144, in forward
    return self.classifier(image_feats_pooled)
  File "/opt/software/install/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/software/install/miniconda3/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 93, in forward
    return F.linear(input, self.weight, self.bias)
  File "/opt/software/install/miniconda3/lib/python3.8/site-packages/torch/nn/functional.py", line 1690, in linear
    ret = torch.addmm(bias, input, weight.t())
RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x4 and 1088x2)

[lixiangchun]

@JihwanEom Thanks for your comments. I still failed to get it work. Appreciate if you could fix the code attached. I haven't applied it to the real world data.

mip_model.txt

[JihwanEom]

@lixiangchun Sorry for removing comments. I'm in the same situation as you..

[JihwanEom]

@lixiangchun Set pooling method from "sum" to "last_timestep", it works.

[anuroopsriram]

Hi @lixiangchun
You are using a batch size of 2, so you need to make sure that you have batch size 2 for times and lens as well.

image_model = DenseNet()
net = MIPModel(image_model, 1024, 64, 2, 2, 1024, 0.1, 0.1, 'sum')
B1, N1, C1, H1, W1 = 2, 4, 3, 224, 224
images = torch.rand(B1, N1, C1, H1, W1)
times = torch.tensor([
    [-30, -20, -10, 0],
    [-30, -20, 0, 0],      # Extra 0 for padding
])
lens = torch.tensor([4, 3])
x = net(images, times, lens)
print(images.shape, x.shape)

Also, make sure to pull the latest master as I made a minor fix.