taking time for inference but returning 'none'

[arunimathappa]

i wanted to test my model. i have trained it on common voice and i get output_graph.pb in the exportdir i created.
on running the following command
[data]$ deepspeech ../../exportdir/output_graph.pb smoke_test/LDC93S1.wav alphabet.txt lm/lm.binary lm/trie
i am getting the following error
Loading model from file ../../exportdir/output_graph.pb
2018-03-26 12:24:04.117793: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: FMA
Loaded model in 0.235s.
Loading language model from files lm/lm.binary lm/trie
Loaded language model in 1.367s.
Running inference.
2018-03-26 12:24:09.156036: E tensorflow/core/framework/op_segment.cc:53] Create kernel failed: Invalid argument: NodeDef mentions attr 'identical_element_shapes' not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=""; is_stateful=true>; NodeDef: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name="bidirectional_rnn/bw/bw/dynamic_rnn/input_0", _device="/job:localhost/replica:0/task:0/device:CPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
2018-03-26 12:24:09.156140: E tensorflow/core/common_runtime/executor.cc:643] Executor failed to create kernel. Invalid argument: NodeDef mentions attr 'identical_element_shapes' not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=""; is_stateful=true>; NodeDef: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name="bidirectional_rnn/bw/bw/dynamic_rnn/input_0", _device="/job:localhost/replica:0/task:0/device:CPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name="bidirectional_rnn/bw/bw/dynamic_rnn/input_0", _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Error running session: Invalid argument: NodeDef mentions attr 'identical_element_shapes' not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=""; is_stateful=true>; NodeDef: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name="bidirectional_rnn/bw/bw/dynamic_rnn/input_0", _device="/job:localhost/replica:0/task:0/device:CPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name="bidirectional_rnn/bw/bw/dynamic_rnn/input_0", _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
None
Inference took 3.452s for 2.925s audio file

what should i do?

[kdavis-mozilla]

Are you using the 0.1.1 version of Deep Speech to train with?

[arunimathappa]

yes. i have used that and also training has been complete. don't know how to proceed with testing

[kdavis-mozilla]

So you trained using the sources in the 0.1.1 release zip, i.e. v0.1.1.zip?

[arunimathappa]

i have trained on common voice using the following command
$ sudo python DeepSpeech.py --train_files ../corpus/cv-valid-train.csv,../corpus/cv-other-train.csv --dev_files ../corpus/cv-valid-dev.csv --test_files ../corpus/cv-valid-test.csv --export_dir ../exportdir --checkpoint_dir ../checkdir _model --train_batch_size 32 --dev_batch_size 30 --test_batch_size 30 --epoch 10 --limit_train 500 --limit_dev 500 --limit_test 500 --display_step 1 --validation_step 1 --learning_rate 0.0001

[kdavis-mozilla]

@arunimathappa What is you use the --one_shot_infer command line argument?

[arunimathappa]

python DeepSpeech.py --one_shot_infer ../../home/clouduser/audio1.wav
for every audio sample it is returning
i the

[kdavis-mozilla]

@arunimathappa Diy you also specify the --checkpoint_dir parameter?

[arunimathappa]

after giving --checkpoint_dir , it is giving me the output but is not that accurate
could you please tell me the command which could do inference through output_graph.pb
i have specified the error above

[kdavis-mozilla]

@arunimathappa Can you answer the template questions we provide for issues? This will help diagnose the problem.

[arunimathappa]

Linux RHEL7
TensorFlow installed from pip command
TensorFlow version : v1.6.0-0-gd2e24b6039 1.6.0
Python version: 3.6
command to reproduce: deepspeech ../exportdirectory/output_graph.pb ../corpus/cv_corpus_v1/cv-other-dev/sample-000000.wav data/alphabet.txt data/lm/lm.binary data/lm/trie
after going inside deepspeech folder

deepspeech version is v0.1.1

error coming is
Loading model from file ../exportdirectory/output_graph.pb
2018-03-28 09:57:35.271674: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: FMA
Loaded model in 0.343s.
Loading language model from files data/lm/lm.binary data/lm/trie
Loaded language model in 1.227s.
Running inference.
2018-03-28 09:57:39.983231: E tensorflow/core/framework/op_segment.cc:53] Create kernel failed: Invalid argument: NodeDef mentions attr 'identical_element_shapes' not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=""; is_stateful=true>; NodeDef: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name="bidirectional_rnn/bw/bw/dynamic_rnn/input_0", _device="/job:localhost/replica:0/task:0/device:CPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
2018-03-28 09:57:39.983332: E tensorflow/core/common_runtime/executor.cc:643] Executor failed to create kernel. Invalid argument: NodeDef mentions attr 'identical_element_shapes' not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=""; is_stateful=true>; NodeDef: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name="bidirectional_rnn/bw/bw/dynamic_rnn/input_0", _device="/job:localhost/replica:0/task:0/device:CPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name="bidirectional_rnn/bw/bw/dynamic_rnn/input_0", _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Error running session: Invalid argument: NodeDef mentions attr 'identical_element_shapes' not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=""; is_stateful=true>; NodeDef: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name="bidirectional_rnn/bw/bw/dynamic_rnn/input_0", _device="/job:localhost/replica:0/task:0/device:CPU:0". (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
[[Node: bidirectional_rnn/bw/bw/TensorArray_1 = TensorArrayV3clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=[?,4096], identical_element_shapes=true, tensor_array_name="bidirectional_rnn/bw/bw/dynamic_rnn/input_0", _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
None
Inference took 3.150s for 3.192s audio file.

it is returning none

how to make it do the inference?
command to train the model was
sudo python DeepSpeech.py --train_files ../corpus/cv-valid-train.csv,../corpus/cv-other-train.csv --dev_files ../corpus/cv-valid-dev.csv --test_files ../corpus/cv-valid-test.csv --export_directory ../exportdir --checkpoint_dir ../checkdirectory _model --train_batch_size 32 --dev_batch_size 30 --test_batch_size 30 --epoch 10 --limit_train 500 --limit_dev 500 --limit_test 500 --display_step 1 --validation_step 1 --learning_rate 0.001

also i am able to do the one_shot_inference

[kdavis-mozilla]

@arunimathappa Did you train the model with 0.1.1 code or current master?

[arunimathappa]

i trained the model with 0.1.1 code

[kdavis-mozilla]

@arunimathappa Why are you using sudo to train the model?

[disha3]

Hi, @arunimathappa and @kdavis-mozilla I am getting exactly this error. Were you able to solve it?

[sandeep-badrinath]

Hi @disha3, @arunimathappa, I am also getting the same error. Were you able to solve this problem?

[sandeep-badrinath]

Using a newer binary for inference worked for me. #1344

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.