tf.Variable will add variable to GLOBAL_VARIABLES, and won't add it to MODEL_VARIABLES. It seem that Superivisor only save MODEL_VARIABLES. I fix this by call tf.contrib.framework.add_model_variable for all varibles defined by tf.Variable.

@shiyemin, Do you mean add following lines:

from __future__ import print_function
import tensorflow as tf

server = tf.train.Server.create_local_server()
logs_path = "mnist/logs"


global_step = tf.get_variable('global_step', [],
                              initializer=tf.constant_initializer(0),
                              trainable=False)
with tf.name_scope("weights"):
    W1 = tf.Variable(tf.random_normal([784, 100]))
    W2 = tf.Variable(tf.random_normal([784, 100]))

for variable in tf.global_variables():
    tf.contrib.framework.add_model_variable(variable)
init_op = tf.global_variables_initializer()
print("Variables initialized ...")
sv = tf.train.Supervisor(is_chief=True,
                         logdir=logs_path,
                         global_step=global_step,
                         init_op=init_op,
                         save_model_secs=600)
with sv.managed_session(server.target) as sess:
    while not sv.should_stop():
        print('==============')
sv.stop()

I tried this, still no success.

I have the same issue - using tf.contrib.framework.add_model_variable(variable) doesn't help.

I encountered a similar error using Tensorflow r0.12 and r1.0 as well. For me, the code breaks when I start a new session using with sv.managed_session(...) as sess. I resolved it by clearing my logs/events in my log directory...

tf.train.Supervisor is deprecated, please use tf.train.MonitoredSession instead. Assigning to @martinwicke since we should mark it deprecated in the docs.

For anyone looking for more information about deprecation of ´tf.trainSupervisor´ and upcoming update to guide, there are some newer comments here: #6604

Can anyone update this example with tf.train.MonitoredSession?

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/dist_test/python/mnist_replica.py

Can anyone update document below with a phrase which clearly indicates that Supervisor is deprecated? Please make Tensorflow community united with clear vision to its API usage.

https://www.tensorflow.org/api_docs/python/tf/train/Supervisor