Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KeyError: u'SaveV2' when loading exported model #5639

Closed
shelpuk opened this issue Nov 16, 2016 · 6 comments
Closed

KeyError: u'SaveV2' when loading exported model #5639

shelpuk opened this issue Nov 16, 2016 · 6 comments
Assignees
@petewarden

Comments

@shelpuk
Copy link

shelpuk commented Nov 16, 2016 •
edited

I have a TensorFLow model trained on a GPU machine. Next, I need to export it and deploy on CPU only production machine.

I have trained and exported a model from a GPU machine as described in MNIST export example. Saver object was initialized above.

with graph.as_default():
    saver = tf.train.Saver(tf.all_variables(), sharded=True)
...

export_path =  'resnet34_rmsprop_wd1e-1/saves/'
print('Exporting trained model to %s' % export_path)
init_op = tf.group(tf.initialize_all_tables(), name='init_op')
model_exporter = exporter.Exporter(saver)
model_exporter.init(sess.graph.as_graph_def(),
                            init_op=init_op,
                            default_graph_signature=exporter.classification_signature(input_tensor=inference_images,
                                                                                      classes_tensor=inference_class,
                                                                                      scores_tensor=inference_predictions),
                            named_graph_signatures={'inputs': exporter.generic_signature({'images': inference_images}),
                                                    'outputs': exporter.generic_signature({'class': inference_class, 'predictions': inference_predictions})})
model_exporter.export(export_path, tf.constant(1), sess)
print('Done exporting!')

Next, I am trying to load saved model to CPU machine with:

new_saver = tf.train.import_meta_graph('assets/saved_model/export.meta') new_saver.restore(sess, 'assets/saved_model/export')

And what I am getting is:

Traceback (most recent call last):
File "script_test_classifier.py", line 4, in <module>
...
line 33, in __initialize_session__
new_saver = tf.train.import_meta_graph('assets/saved_model/export.meta')
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1711, in import_meta_graph
read_meta_graph_file(meta_graph_or_file), clear_devices)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1598, in _import_meta_graph_def
input_graph_def, name="", producer_op_list=producer_op_list)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/importer.py", line 258, in import_graph_def
op_def = op_dict[node.op]
KeyError: u'SaveV2'

What is the reason of the error and how it could be fixed?

Production (CPU machine) environment info:
AWS instance type: m4.xlarge
Operating System: Ubuntu 14.04 x64
Installed version of CUDA and cuDNN: None, I am using CPU version of TF for the production environment
A link to the pip package you installed: https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0-cp27-none-linux_x86_64.whl
The output from python -c "import tensorflow; print(tensorflow.__version__)": 0.11.0
@girving
Copy link
Contributor

girving commented Nov 16, 2016

@shelpuk I believe you are using a newer version of TensorFlow to make the graph, which then isn't readable by the older version of TensorFlow on the production machine. TensorFlow is not guaranteed to be forward compatible across versions.

@petewarden Looks like this may be your importer code. Throwing an exception is correct, but KeyError isn't a very informative exception. You up for cleaning that up?

@dhwajraj dhwajraj mentioned this issue Dec 2, 2016
Closed
@jart
Copy link
Contributor

jart commented Jan 16, 2017

Closing due to inactivity. Most likely due to needing a newer version of TensorFlow installed. Also see related commit improving error message.

@jart jart closed this as completed Jan 16, 2017
@4emkay
Copy link

4emkay commented Mar 20, 2018

/home/pr/PycharmProjects/TensorVoice/venv/bin/python /home/pr/Documents/projects/speech_recognition/Speech_Commands/test.py
2018-03-20 16:39:54.772431: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Traceback (most recent call last):
File "/home/pr/Documents/projects/speech_recognition/Speech_Commands/test.py", line 41, in
new_saver = tf.train.import_meta_graph('/home/pr/Documents/projects/speech_recognition/Speech_Commands/trains/conv.ckpt-8.meta')
File "/home/pr/PycharmProjects/TensorVoice/venv/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1909, in import_meta_graph
**kwargs)
File "/home/pr/PycharmProjects/TensorVoice/venv/lib/python3.5/site-packages/tensorflow/python/framework/meta_graph.py", line 737, in import_scoped_meta_graph
producer_op_list=producer_op_list)
File "/home/pr/PycharmProjects/TensorVoice/venv/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 432, in new_func
return func(*args, **kwargs)
File "/home/pr/PycharmProjects/TensorVoice/venv/lib/python3.5/site-packages/tensorflow/python/framework/importer.py", line 431, in import_graph_def
_RemoveDefaultAttrs(op_dict, producer_op_list, graph_def)
File "/home/pr/PycharmProjects/TensorVoice/venv/lib/python3.5/site-packages/tensorflow/python/framework/importer.py", line 211, in _RemoveDefaultAttrs
op_def = op_dict[node.op]
KeyError: # 'DecodeWav'

Even i am getting the same problem, Do i have to degrade my Tensorflow

??

@iamgroot42
Copy link

iamgroot42 commented Jul 13, 2018 •
edited

KeyErrors in loading a Tensorflow graph can be because of missing explicit import statements. I ran into the same error for no op named GatherTree. Apparently Tensorflow does some dynamic loading of imports, thus causing such errors. In my opinion, this is more of a bug and at least the error messages should suggest an appropriate action. Anyway, I fixed my issue using the following:

from tensorflow.contrib.seq2seq.python.ops import beam_search_ops

You can try a similar solution by looking up the relevant op that should be imported.

@tamizharasank
Copy link

KeyError: u'SSTableReaderV2'

@ontheklaud
Copy link

I also experienced "KeyError: 'CSVDataset'" while loading protobuf (saved_model.pb) thru tf.saved_model.loader.load.
After declaring 'from tensorflow.contrib.data import CsvDataset', my app works like a charm!
Thx @iamgroot42 :) 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@petewarden petewarden

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@jart @girving @petewarden @shelpuk @iamgroot42 @tamizharasank @ontheklaud @4emkay