eval_transducer.sh decoder error

[bliunlpr2020]

When I run eval_transducer.sh, got the error:

Traceback (most recent call last):
File "/work0/pika//decoder/decode_transducer.py", line 264, in
main()
File "/work0/pika//decoder/decode_transducer.py", line 133, in main
len_batch + 100)
File "/work0/pika/decoder/transducer_decoder.py", line 182, in decode_batch
b.advance(out[:, j], self.t_idx[:, j], x_len[j])
File "/work0/pika/decoder/beam_transducer.py", line 140, in advance
self.scores -= self.lm_scorer_scale * self.lm_scores[prev_k]
IndexError: tensors used as indices must be long, byte or bool tensors

And I print some outputs:
beam_scores tensor([ -0.6430, -12.0813, -6.1094, ..., -8.2931, -11.2605, -11.8402],
device='cuda:0', grad_fn=) torch.Size([4232])
flat_beam_scores tensor([ -0.6430, -12.0813, -6.1094, ..., -8.2931, -11.2605, -11.8402],
device='cuda:0', grad_fn=) torch.Size([4232])
best_scores_id tensor([ 0, 156, 2995, 3703, 30, 718, 850, 1150], device='cuda:0') torch.Size([8]) <class 'torch.Tensor'>
num_words 4232 <class 'int'>
prev_k tensor([0.0000, 0.0369, 0.7077, 0.8750, 0.0071, 0.1697, 0.2009, 0.2717],
device='cuda:0') <class 'torch.Tensor'>

I use the pytorch 1.7.1.

Any suggestions? Thanks!

[cweng6]

It seems / or // is deprecated for integer division in pytorch 1.7.0, could you try replacing this line (

pika/decoder/beam_transducer.py

Line 125 in 23c4cdd

prev_k = best_scores_id / num_words

)

prev_k = best_scores_id / num_words

to,

prev_k = torch.floor_divide(best_scores_id, num_words)

should solve your problem. I am not sure if this change will affect back compatibility with the older PyTorch version. Will submit a PR once confirmed.

[cweng6]

@bliunlpr2020 please let us know if your issue has been fixed or not.

[bliunlpr2020]

My issue has been fixed. Thank you very much! Maybe eval_transducer.sh miss "--cmn" .

[cweng6]

It seems so. As long as '--cmn' option complies with the one used in the compute_cmvn_stat.py script, you should be good. If the issue has been fixed, could you please close it. Thanks.