Want bair code

[rastaman7]

This is related to the former question "Want moving_mnist code"(#1) by ManWingloeng.

Could you please offer us bair.py as well?
We would like to test using RobotPush dataset.

This code is from data_utils.py,

from data.moving_mnist import MovingMNIST
from data.bair import RobotPush
from data.kth128 import KTH

[gnosisyuw]

You can find the dataloader and data preprocessing code for BAIR in the following link
https://github.com/edenton/svg/tree/master/data
Note that the architectures used for BAIR is different from the one used for Moving MNIST.

[rastaman7]

Thank you very much for sharing the bair code.
I just want to confirm, when you say the architecture is different, you mean the dataset (and not the algorithm)?

[gnosisyuw]

Thank you very much for sharing the bair code.
I just want to confirm, when you say the architecture is different, you mean the dataset (and not the algorithm)?

No. For BAIR, the task is stochastic video generation instead of video prediction. So the model architecture is a combination of SVG and CrevNet which was not reported in the paper. The pipeline looks like below.

[rastaman7]

Thank you for the response.
We are currently running into a problem when using bair dataset.
We get an error when trying to train using --channels=3.
This is the steps we took.

① Using download_bair.sh from the link you sent us, we got the tar file and decompressed it.
② Using convert_bair.py, we extracted the images from tfrecords.
③After changing some arguments on model_mnist.py (--dataset and --data_root), we ran model_mnist.py
④ We got images like the ones below (in black and white)
⑤ If we change the number of channels to 3 (--channels=3), we get this error.

Traceback (most recent call last):
File "model_mnist.py", line 264, in
mse = train(input,epoch)
File "model_mnist.py", line 232, in train
h = encoder(x[i - 1], True)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/mnt/VideoPrediction/CrevNet-Traffic4cast/layers_3d.py", line 124, in forward
out = block.forward(out)
File "/mnt/VideoPrediction/CrevNet-Traffic4cast/layers_3d.py", line 51, in forward
Fx2 = self.bottleneck_block(x2)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 448, in forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [3, 6, 3, 3, 3], expected input[16, 0, 3, 32, 32] to have 6 channels, but got 0 channels instead

How can we solve this?
Thank you very much for helping us.

[rrxi]

@rastaman7 , I get same problem with you, when i set channel to 3. Do you resolve this problem？Thanks.

[rastaman7]

@rrxi , No, we haven't been able to figure this out yet...

[Mareeta26]

@rastaman7 , @rrxi You can set the number of hidden layers - rnn_size as 96, to match the output feature size from the encoder.