New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

freeze_graph inspect_checkpoint, Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator #7696

Closed
apiszcz opened this Issue Feb 20, 2017 · 6 comments

Comments

Projects
None yet
8 participants
@apiszcz

apiszcz commented Feb 20, 2017

What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?

Nothing directly applicable to the observed error.

Environment info

TF 0.12.1, Windows 10/64, Python 3.5/64

Installed version of CUDA and cuDNN:
8.0, 5.1

If possible, provide a minimal reproducible example (We usually don't have time to read hundreds of lines of your code)

inspect_checkpoint

lib\python\python lib\tensorflow_cpu\tensorflow\python\tools\inspect_checkpoint.py --file_name model.ckpt-250514.data-00000-of-00001 --tensor_name-''

2017-02-20 08:23:19.291800: W c:\tf_jenkins\home\workspace\nightly-win\device\cpu\os\windows\tensorflow\core\util\tensor_slice_reader.cc:95] Could not open model.ckpt-250514.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
Unable to open table file model.ckpt-250514.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

freeze_graph

Similar error
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file model.ckpt-250514.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
[[Node: save/RestoreV2_445 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_445/tensor_names, save/RestoreV2_445/shape_and_slices)]]

DataLossError (see above for traceback): Unable to open table file C:\g\vx\data\jhole\model\model.ckpt-250514.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
[[Node: save/RestoreV2_780 = RestoreV2[dtypes=[DT_INT64], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_780/tensor_names, save/RestoreV2_780/shape_and_slices)]]

What other attempted solutions have you tried?

  1. A few different checkpoint files.
  2. Tried freeze_graph using TF 1.0, same result

Logs or other output that would be helpful

Messages similar "Similar error" above is repeated many times.

@yaroslavvb

This comment has been minimized.

Show comment
Hide comment
Contributor

yaroslavvb commented Feb 20, 2017

@aselle

This comment has been minimized.

Show comment
Hide comment
@aselle

aselle Mar 3, 2017

Member

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

Member

aselle commented Mar 3, 2017

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

@aselle aselle closed this Mar 3, 2017

@AnranQi

This comment has been minimized.

Show comment
Hide comment
@AnranQi

AnranQi Jul 25, 2017

Anyone who solve this problem?

AnranQi commented Jul 25, 2017

Anyone who solve this problem?

@tangyanlin

This comment has been minimized.

Show comment
Hide comment
@tangyanlin

tangyanlin Aug 19, 2017

just regard the prefix as checkpoint, like:
lib\python\python lib\tensorflow_cpu\tensorflow\python\tools\inspect_checkpoint.py --file_name model.ckpt-250514

tangyanlin commented Aug 19, 2017

just regard the prefix as checkpoint, like:
lib\python\python lib\tensorflow_cpu\tensorflow\python\tools\inspect_checkpoint.py --file_name model.ckpt-250514

@LingjiaDeng

This comment has been minimized.

Show comment
Hide comment
@LingjiaDeng

LingjiaDeng Sep 1, 2017

I saved the model using V2 format, which gave me three files:

  • model-10000.data-00000-of-00001
  • model-10000.index
  • model-10000.meta

As @tangyanlin metioned, use the prefix. My input parameter is --input_checkpoint=checkpoints/model-10000 (NOT including the period) which works.

LingjiaDeng commented Sep 1, 2017

I saved the model using V2 format, which gave me three files:

  • model-10000.data-00000-of-00001
  • model-10000.index
  • model-10000.meta

As @tangyanlin metioned, use the prefix. My input parameter is --input_checkpoint=checkpoints/model-10000 (NOT including the period) which works.

@zhangchunsheng

This comment has been minimized.

Show comment
Hide comment
@zhangchunsheng

zhangchunsheng Nov 19, 2017

Get this when I train model

INFO:tensorflow:Summary name /clone_loss is illegal; using clone_loss instead.
/Library/Python/2.7/site-packages/tensorflow/python/ops/gradients_impl.py:96: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "

then can't freeze model

python object_detection/export_inference_graph.py --input_type image_tensor --pipeline_config_path models/faster_rcnn_numobj.config --trained_checkpoint_prefix models/train/model.ckpt-* --output_directory models/result/

2017-11-19 22:26:38.039878: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open train/model.ckpt-0.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

zhangchunsheng commented Nov 19, 2017

Get this when I train model

INFO:tensorflow:Summary name /clone_loss is illegal; using clone_loss instead.
/Library/Python/2.7/site-packages/tensorflow/python/ops/gradients_impl.py:96: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "

then can't freeze model

python object_detection/export_inference_graph.py --input_type image_tensor --pipeline_config_path models/faster_rcnn_numobj.config --trained_checkpoint_prefix models/train/model.ckpt-* --output_directory models/result/

2017-11-19 22:26:38.039878: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open train/model.ckpt-0.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment