Function call stack: _test_function -> _test_function #97
Description
Hi,
my env:
Anaconda3
python == 3.8
tensorflow-gpu == 2.3
I try to run test_streaming_transducer.py --saved model at TensorASR-0.6.3. And encounter error
2021-01-05 16:06:31.453448: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at scatter_nd_op.cc:212 : Invalid argument: indices[0] = [183] does not index into shape [183]
Traceback (most recent call last):
File "examples/streaming_transducer/test_streaming_transducer.py", line 100, in <module>
streaming_transducer_tester.run(test_dataset)
File "/home/user/anaconda3/envs/tfasr/lib/python3.8/site-packages/tensorflow_asr/runners/base_runners.py", line 406, in run
self._test_epoch()
File "/home/user/anaconda3/envs/tfasr/lib/python3.8/site-packages/tensorflow_asr/runners/base_runners.py", line 417, in _test_epoch
decoded = self._test_function(test_iter)
File "/home/user/anaconda3/envs/tfasr/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 780, in __call__
result = self._call(*args, **kwds)
File "/home/user/anaconda3/envs/tfasr/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 814, in _call
results = self._stateful_fn(*args, **kwds)
File "/home/user/anaconda3/envs/tfasr/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2829, in __call__
return graph_function._filtered_call(args, kwargs) # pylint: disable=protected-access
File "/home/user/anaconda3/envs/tfasr/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1843, in _filtered_call
return self._call_flat(
File "/home/user/anaconda3/envs/tfasr/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1923, in _call_flat
return self._build_call_outputs(self._inference_function.call(
File "/home/user/anaconda3/envs/tfasr/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 545, in call
outputs = execute.execute(
File "/home/user/anaconda3/envs/tfasr/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: indices[0] = [183] does not index into shape [183]
[[{{node StatefulPartitionedCall/StatefulPartitionedCall_1/while/body/_309/while/streaming_transducer_beam_search/while/body/_613/while/streaming_transducer_beam_search/while/while/body/_891/while/streaming_transducer_beam_search/while/while/while/body/_1119/while/streaming_transducer_beam_search/while/while/while/cond/else/_1289/while/streaming_transducer_beam_search/while/while/while/cond/TensorScatterUpdate}}]]
(1) Invalid argument: indices[0] = [183] does not index into shape [183]
[[{{node StatefulPartitionedCall/StatefulPartitionedCall_1/while/body/_309/while/streaming_transducer_beam_search/while/body/_613/while/streaming_transducer_beam_search/while/while/body/_891/while/strea
ming_transducer_beam_search/while/while/while/body/_1119/while/streaming_transducer_beam_search/while/while/while/cond/else/_1289/while/streaming_transducer_beam_search/while/while/while/cond/TensorScatterUpdate}}]]
[[StatefulPartitionedCall/StatefulPartitionedCall_1/while/body/_309/while/streaming_transducer_beam_search/while/body/_613/while/streaming_transducer_beam_search/while/while/body/_891/while/streaming_transducer_beam_search/while/while/while/body/_1119/while/streaming_transducer_beam_search/while/while/while/cond/else/_1289/while/streaming_transducer_beam_search/while/while/while/cond/TensorScatterUpdate/_192]]0 successful operations.
0 derived errors ignored. [Op:__inference__test_function_21087]
Function call stack:
_test_function -> _test_function
At first, I encounter ValueError: Dimension 0 in both shapes must be equal, but are 2 and 1. Shapes are [2,1024] and [1,1024].
So I fix runing_config.batch_size: 1 in config.yml.
Then It work until 2% (21/1310) (libri/test-clean.tsv). After that, The error occur
How can I fix it ?
Thanks