使用tf.train.shuffle_batch方法出现异常

[superping]

使用caicloud.clever.tensorflow库在本机运行的，不带caicloud.clever.tensorflow库的代码原本可用
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: You must feed a value for placeholder tensor 'image' with dtype float and shape [12,224,224,3]
[[Node: image = Placeholderdtype=DT_FLOAT, shape=[12,224,224,3], _device="/job:localhost/replica:0/task:0/cpu:0"]]
Traceback (most recent call last):
File "./src/train.py", line 256, in
distTfRunner.run(train_fn)
File "/usr/local/lib/python2.7/site-packages/caicloud/clever/tensorflow/dist_base.py", line 250, in run
should_stop = train_fn(sess, step)
File "./src/train.py", line 166, in train_fn
images_array, labels_array = session.run([_train_images, _train_labels])
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 767, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 12, current size 1)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

Caused by op u'shuffle_batch', defined at:
File "./src/train.py", line 256, in
distTfRunner.run(train_fn)
File "/usr/local/lib/python2.7/site-packages/caicloud/clever/tensorflow/dist_base.py", line 208, in run
model_fn_handler = self._call_model_fn()
File "/usr/local/lib/python2.7/site-packages/caicloud/clever/tensorflow/dist_base.py", line 169, in _call_model_fn
model_fn_handler = self._model_fn(False, 1)
File "./src/train.py", line 83, in model_fn
_train_images, _train_labels = inputs(FLAGS.batch, FLAGS.train, FLAGS.train_labels)
File "/Users/xieanping/sourcecode/github/segnet-caicloud/src/inputs.py", line 29, in inputs
min_after_dequeue=500)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 1165, in shuffle_batch
name=name)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 739, in _shuffle_batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 458, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 1310, in _queue_dequeue_many_v2
timeout_ms=timeout_ms, name=name)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2327, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1226, in init
self._traceback = _extract_stack()

OutOfRangeError (see above for traceback): RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 12, current size 1)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

[lienhua34]

@superping 能否提供一下 src/train.py 文件的代码，我看异常信息是，你定义了一个 placeholder，但是在 session.run() 的时候没有 feed。

[lienhua34]

@superping
这个问题看了一下，这个是目前平台的一个 Bug，当 tf.summary 中要保存的数据需要进行 feed 的时候，就会出现这个问题。这个问题我们纪录下来，后续版本会进行修复。多谢反馈！

目前一个规避措施：在 ｀model_fn｀ 返回的 ModelFnHandler 对象中将 summary_op 设置为 None，让
TaaS 平台不自动计算 summary 信息。

[caicloud-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[caicloud-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[caicloud-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close