Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError: Length of all samples has to be greater than 0 in validation set #1342

Closed
lizamiller opened this Issue Mar 5, 2019 · 3 comments

Comments

Projects
None yet
2 participants
@lizamiller
Copy link

commented Mar 5, 2019

Loading dataset from dataset/data.train.0.pt, number of examples: 93871
the process fails once i get to
[2019-03-05 12:34:01,506 INFO] Loading dataset from dataset/data.valid.0.pt, number of examples: 20000
and this is the error i get

Traceback (most recent call last):
File "train.py", line 109, in
main(opt)
File "train.py", line 39, in main
single_main(opt, 0)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/train_single.py", line 116, in main
valid_steps=opt.valid_steps)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/trainer.py", line 224, in train
valid_iter, moving_average=self.moving_average)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/trainer.py", line 274, in validate
outputs, attns = valid_model(src, tgt, src_lengths)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/models/model.py", line 42, in forward
enc_state, memory_bank, lengths = self.encoder(src, lengths)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/encoders/rnn_encoder.py", line 74, in forward
packed_emb = pack(emb, lengths_list)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/utils/rnn.py", line 148, in pack_padded_sequence
return PackedSequence(torch._C._VariableFunctions._pack_padded_sequence(input, lengths, batch_first))
RuntimeError: Length of all samples has to be greater than 0, but found an element in 'lengths' that is <= 0

should i define somewhere validation size? as it is much smaller than the regular size

@lizamiller

This comment has been minimized.

Copy link
Author

commented Mar 5, 2019

I am using the last OpenNMT-py (git clomed today)
the process fails once it gets to validation
any way to define validation size without running into the error
thanks

@vince62s

This comment has been minimized.

Copy link
Contributor

commented Mar 5, 2019

you must have an empty line in the valid set.

@lizamiller

This comment has been minimized.

Copy link
Author

commented Mar 5, 2019

I didn't find an empty line but i had /p that must have caused the error.
Thanks

@vince62s vince62s closed this Mar 5, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.