Runtime error

[shaform]

conda create -n pytorch python=3.5 anaconda
source activate pytorch
conda install pytorch torchvision cuda80 -c soumith
python train.py --data_path PATH_TO_PROCESSED_DATA --cuda --kenlm_path PATH_TO_KENLM_DIRECTORY

Training...
Traceback (most recent call last):
File "train.py", line 516, in
train_ae(train_data[niter], total_loss_ae, start_time, niter)
File "train.py", line 352, in train_ae
output = autoencoder(source, lengths, noise=True)
File "/home/$USER/anaconda2/envs/pytorch/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in call
result = self.forward(*input, **kwargs)
File "/home/$USER/workspace/ARAE/pytorch/models.py", line 174, in forward
hidden = self.encode(indices, lengths, noise)
File "/home/$USER/workspace/ARAE/pytorch/models.py", line 202, in encode
hidden = torch.div(hidden, norms.expand_as(hidden))
File "/home/$USER/anaconda2/envs/pytorch/lib/python3.5/site-packages/torch/autograd/variable.py", line 725, in expand_as
return Expand.apply(self, (tensor.size(),))
File "/home/$USER/anaconda2/envs/pytorch/lib/python3.5/site-packages/torch/autograd/_functions/tensor.py", line 111, in forward
result = i.expand(*new_size)
RuntimeError: The expanded size of the tensor (300) must match the existing size (64) at non-singleton dimension 1. at /opt/conda/conda-bld/pytorch_1502008109146/work/torch/lib/THC/generic/THCTensor.c:323

[wj926]

I solved this problem by using --batch_size 300

[kellywzhang]

Hello! So I've also run into the same issue and it seems to depend on the version of PyTorch that you're using. So the code at line 202 in models.py hidden = torch.div(hidden, norms.expand_as(hidden)) worked in the last major release version of PyTorch. For the newest release of PyTorch I've had to change it to hidden = torch.div(hidden, norms.unsqueeze(1).expand_as(hidden)).

Basically this section of the code finds the L2 norm for each of the hidden vectors / codes in the batch, and then divides the hidden vectors by the L2 norm to normalize them into unit vectors. The problem is just that of PyTorch syntax in changing the dimension of the norm to prepare it for the division.

Let me know if you have any more issues related to this.

[shaform]

Thanks, this indeed solved the problem.
For reference, in v0.1.12 torch.norm always keep dims:

The output Tensor is of the same size as input except in the dimension dim where it is of size 1.
http://pytorch.org/docs/0.1.12/torch.html?highlight=norm#torch.norm

But in v0.2.0, the behavior was changed:

If keepdim is true, the output Tensor is of the same size as input except in the dimension dim where it is of size 1. Otherwise, dim is squeezed.
http://pytorch.org/docs/0.2.0/torch.html?highlight=norm#torch.norm

[JulesGM]

#9 (comment) I think this should be uncommented by default now.. it's been almost a year