Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why doesn't tensorflow support tensor as the feed_dict? #3389

Closed
wishforgood opened this issue Jul 19, 2016 · 16 comments
Closed

Why doesn't tensorflow support tensor as the feed_dict? #3389

wishforgood opened this issue Jul 19, 2016 · 16 comments

Comments

@wishforgood
Copy link

wishforgood commented Jul 19, 2016

What I'm trying to do

I am trying to extract CNN features for my own images with residual-net based on https://github.com/ry/tensorflow-resnet. I plan to input image data from JPG files before exploring how to convert the images into a single file.

What I have done

I have read https://www.tensorflow.org/versions/r0.9/how_tos/reading_data/index.html and some related materials about how to input data like feeding and placeholder. Here is my code:

import tensorflow as tf
from convert import print_prob, checkpoint_fn, meta_fn
from image_processing import image_preprocessing
tf.app.flags.DEFINE_integer('batch_size', 1, "batch size")
tf.app.flags.DEFINE_integer('input_size', 224, "input image size")
tf.app.flags.DEFINE_integer('min_after_dequeue', 224, "min after dequeue")
tf.app.flags.DEFINE_integer('layers', 152, "The number of layers in the net")
tf.app.flags.DEFINE_integer('image_number', 6951, "number of images")
FLAGS = tf.app.flags.FLAGS


def placeholder_inputs():
    images_placeholder = tf.placeholder(tf.float32, shape=(FLAGS.batch_size, FLAGS.input_size, FLAGS.input_size, 3))
    label_placeholder = tf.placeholder(tf.int32, shape=FLAGS.batch_size)
    return images_placeholder, label_placeholder


def fill_feed_dict(image_ba, label_ba, images_pl, labels_pl):
    feed_dict = {
        images_pl: image_ba,
    }
    return feed_dict

min_fraction_of_examples_in_queue = 0.4
min_queue_examples = int(FLAGS.image_number *
                     min_fraction_of_examples_in_queue)
dataset = tf.train.string_input_producer(["hollywood_test.txt"])
reader = tf.TextLineReader()
_, file_content = reader.read(dataset)
image_name, label, _ = tf.decode_csv(file_content, [[""], [""], [""]], " ")
label = tf.string_to_number(label)
num_preprocess_threads = 10
images_and_labels = []
with tf.Session() as sess:
    for thread_id in range(num_preprocess_threads):
        image_buffer = tf.read_file(image_name)
        bbox = []
        train = False
        image = image_preprocessing(image_buffer, bbox, train, thread_id)
        image = image_buffer
        images_and_labels.append([image, label])
    image_batch, label_batch = tf.train.batch_join(images_and_labels,
                                            batch_size=FLAGS.batch_size,
                                            capacity=min_queue_examples + 3 * FLAGS.batch_size)
    images_placeholder, labels_placeholder = placeholder_inputs()
    new_saver = tf.train.import_meta_graph(meta_fn(FLAGS.layers))
    new_saver.restore(sess, checkpoint_fn(FLAGS.layers))
    graph = tf.get_default_graph()
    prob_tensor = graph.get_tensor_by_name("prob:0")
    images = graph.get_tensor_by_name("images:0")
    feed_dict = fill_feed_dict(image_batch, label_batch, images, labels_placeholder)
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)
    sess.run(tf.initialize_all_variables())
    prob = sess.run(prob_tensor, feed_dict=feed_dict)
    print_prob(prob[0])
    coord.request_stop()
    coord.join(threads)

What my question is

The code above got the error TypeError: The value of a feed cannot be a tf.Tensor object. Acceptable feed values include Python scalars, strings, lists, or numpy ndarrays. You can see I am trying to do all the work in the context of tensorflow. As far as I know, tensorflow is a framework where all the inputs and outputs of the nodes are tensors. However, I am quite confused why feed_dict doesn't support tensor as an input. But the batch_join function return a tensor and so do other ops. I found that in the mnist example of tensorflow there is even another function to produce a batch when tensorflow is providing methods for batching. So I wonder if there is an elegant way to do these things. If this is because of my lack of searching and careful reading I really apologize.

@zakizhou
Copy link

zakizhou commented Jul 19, 2016

I'm new to tensorflow, according to my experience, you really don't need to feed a tf.tensor to feed_dict. Once you have a tensor with value, you can get this value with your_tensor.eval() or sess.run(your_tensor) and then you can feed the output to your feed_dict

@wishforgood
Copy link
Author

wishforgood commented Jul 19, 2016

I have also searched about how to get the value of a tensor to solve this problem and seen the solutions like above. However when I try to get image_batch.eval() or even image_buffer.eval(), the program just keep working without stopping or giving any output. I have tried to get a constant value from a tensor and succeeded. But this is just useless with tensor gotten from tf.read_file() and etc.

@fwalch
Copy link

fwalch commented Jul 19, 2016

If you have your data in tensors anyway, you don't need to use placeholders and feed_dict. You can just directly use image_batch / label_batch instead of images_placeholder / label_placeholder when setting up your model / loss.

@wishforgood
Copy link
Author

Yes, I can see from the examples of training procedures that I can just use tensors as input of any ops in tensorflow. However in this case I am using a predefined network so I am confused whether I can change the network definition. That is to say, I have no idea how to change the input of loss function(the definition of node "images" and "prob" is images = tf.placeholder("float32", [None, 224, 224, 3], name="images") logits = resnet.inference(images, is_training=False, num_blocks=num_blocks, preprocess=True, bottleneck=True) prob = tf.nn.softmax(logits, name='prob') in the predefined model).

@concretevitamin
Copy link
Contributor

StackOverflow is a better venue. One key thing to note is that Tensor is simply a symbolic object. The values of your feed_dict are the actual values, e.g. a Numpy ndarry.

@CSGrandeur
Copy link

CSGrandeur commented Sep 23, 2016

I have the same questions with @wishforgood .
"I have no idea how to change the input of loss function"...

How?

@olegklimov
Copy link

Why is this closed?

Author raises legitimate concerns.

  1. Why data already on GPU, cannot be fed into GPU model.

  2. Why redefine model to use data already on GPU. This makes code more complicated.

  3. What if I have 5 variables with data, do I need to define now 5 models with shared trainable variables?

Yes it is confusing, make you write complicated (or slow) code -- I think you guys should reopen and address this.

@Kwoks2017
Copy link

I have the same opinion as yours. These days, i use the cnn to do some text work. For the input's dimension is too large, i have to change it to other type in case of the memory error. But , what confused me is the same as yous. Why tensor cannot be a feed_dict value!!!

@vsubhashini
Copy link

Looks like you could use a session_handle to do this:

Simple case:

p = tf.placeholder(tf.float32, shape=()) 
c = tf.constant(1.0, dtype=tf.float32)
h = tf.get_session_handle(c)
h = sess.run(h)        # gives you a handle
sess.run(p, feed_dict={p:h})

The below is an example from the get_session_handle doc:

c = tf.multiply(v1, v2)
h = tf.get_session_handle(c)
h = sess.run(h)        # gives you a handle

p, a = tf.get_session_tensor(h.handle, tf.float32)
b = tf.multiply(a, 10)
c = sess.run(b, feed_dict={p: h.handle})

@anilesec
Copy link

This works for a scalar value but when you try to get handle for a sequence of inputs, it is failing.
Basically, i am also getting the same error "TypeError: The value of a feed cannot be a tf.Tensor object. Acceptable feed values include Python scalars, strings, lists, numpy ndarrays, or TensorHandles."

operation I performed is feeding the sequence of number from -10 to 10 for a function.

part of code:
f = tf.square(x) + 2 * x + 5

val_x = tf.range(-10, 10, 0.1)

val_f = sess.run(f, feed_dict = {x: val_x})

But if i replace val_x = tf.range(-10, 10, 0.1) --> val_x = np.arange(-10, 10, 0.1), it works fine without any errors.

why is sequence generated using tf.range() is not treated as tensor object?

Thanks in Advance!

@Kwoks2017
Copy link

@Anilkumares For the feed_dict can't accept the tensor object. As for the f = tf.square(x) + 2 * x + 5, it can be fed with scalars generated by numpy

@pranka02
Copy link

pranka02 commented Dec 23, 2017

You can do something like this in a loop to make batches without using any TensorFlow backend. This will ensure that the output of it remains non-tensor.

with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
            for index, offset in enumerate(range(0, train, batch_size)):
                x_epoch, y_epoch = np.array(train_x[offset: offset + batch_size,:]), np.array(train_y[offset: offset +  batch_size])

@Arsey
Copy link

Arsey commented Apr 20, 2018

The short answer to the question is because of Tensorflow makeы simple things complicated

@SystemErrorWang
Copy link

I may have a different question but similar with yours: if my input data is already in tensor type, how can i feed them into the network? this is very simple and straightforward in pytorch, but as we know, tensorflow uses static computational graph. we have to pre-define the graph and then change the input in a cycle. if the inputs of network and loss function are defined when creating the graph, how can we change it during training?

@lakrish
Copy link

lakrish commented Apr 19, 2019

One workaround is to construct a graph def and load it in with input_mapping, through this the tf.data.Iterator can be hooked up to any arbitrary graph which previously used placeholders for input. This is especially beneficial for inference, since weights can be frozen.

@davidparks21
Copy link

davidparks21 commented Jul 12, 2019

For anyone who lands on this question, Lakrish provided a very usable solution (only a few lines of code). I have posted an example using this suggestion at this related SO question: https://stackoverflow.com/questions/38618960/tensorflow-how-to-insert-custom-input-to-existing-graph/57015133#57015133

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests