Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dimensionality error on GPU for changed number of encoder layers #101

Open
dieuwkehupkes opened this issue Nov 20, 2017 · 3 comments
Open

Comments

@dieuwkehupkes
Copy link

I am encountering a strange problem when I try to change the number of layers in the encoder: when I run this on a cpu, it runs without problems, when I call the exact same script on a gpu, however it gives me a dimensionality error. The only thing I changed is the call of the decoder in the sample.py script:

encoder = EncoderRNN(len(src.vocab), max_len, hidden_size, n_layers=2, bidirectional=bidirectional, variable_lengths=True)

Which results in the following error when forward is called:

File "/home/dhupkes/.local/lib/python2.7/site-packages/torch/backends/cudnn/rnn.py", line 266, in forward hidden_size, tuple(hx.size()))) RuntimeError: Expected hidden size (1, 32L, 256), got (2L, 32L, 256L)

I imagined that this would be due to what is passed to the decoder, but when I started to debug on a cpu I discovered to my surprise that the error was not raised there with the exact same script.

Anyone an idea what is going on?

@tejaswini
Copy link
Contributor

Hi,
Please include n_layers=2 in the decoder too. That seems to have fixed the issue.
encoder = EncoderRNN(len(src.vocab), max_len, hidden_size, n_layers=2,
bidirectional=bidirectional, variable_lengths=True)
decoder = DecoderRNN(len(tgt.vocab), max_len, hidden_size * 2 if bidirectional else 1,
n_layers=2, dropout_p=0.2, use_attention=True,
bidirectional=bidirectional, eos_id=tgt.eos_id,
sos_id=tgt.sos_id)

@tejaswini tejaswini self-assigned this Nov 25, 2017
@dieuwkehupkes
Copy link
Author

Hey,
Thanks for looking into it. Did you run this also on a GPU? Because I tried this already but it didn't solve the problem (well, no errors on the CPU, but only on GPU). Why do you think this should help? I don't think there is any theoretical reason that the encoder should have the same number of layers as the decoder (and it is in fact also not what I would like to have in my model).

@tejaswini
Copy link
Contributor

Hi,
Turns out the discrepancy in the behaviour between CPU and GPU is a bug in pytorch. We filed a bug for the same pytorch/pytorch#4002
Currently, we only support encoders and decoders with the same number of hidden layers. We created an issue to support what you are doing #107.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants