Try to call tf.select but get AttributeError: 'module' object has no attribute 'select'

[zehzhang]

Here is the traceback information:
Traceback (most recent call last):
File "guidedBack.py", line 149, in
grads = K.gradients(model.layers[-2].output[0, 0], model.layers[-6].layers[-2].output)[0]
File "/N/u/zehzhang/myTensorflow/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 2108, in gradients
return tf.gradients(loss, variables, colocate_gradients_with_ops=True)
File "/N/u/zehzhang/myTensorflow/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py", line 482, in gradients
in_grads = grad_fn(op, *out_grads)
File "guidedBack.py", line 25, in _GuidedReluGrad
return tf.select(0. < grad, gen_nn_ops._relu_grad(grad, op.outputs[0]), tf.zeros(grad.get_shape()[1:]))
AttributeError: 'module' object has no attribute 'select'

Anyone has any idea?

[jubjamie]

Is Tensorflow imported correctly as tf in the file throwing the error?
Does the top of that file read:

import tensorflow as tf

Please check your import statement.

[zehzhang]

@jubjamie
Yes, I have imported tensorflow as tf.

[jubjamie]

Can you please post the code of the file causing problems? I will try and reproduce the problem.

[zehzhang]

I found in the new version of tensorflow, there was one file named select.py added to tensorflow.contrib.graph_editor and it seems tensorflow.contrib is lazily imported in init.py.

Is this the reason?

[zehzhang]

@jubjamie
Here it is (still some cleaning work to do, but enough for debugging :) ):

from keras.applications.vgg16 import VGG16, preprocess_input
from keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array
from keras.layers import Activation, Dropout, Flatten, Dense, Lambda
from keras.models import Model, Sequential
import numpy as np
from scipy.misc import imresize
from scipy.misc import imsave
import tensorflow as tf
import keras.backend as K
from tensorflow.python.framework import ops
from tensorflow.python.ops import gen_nn_ops


@ops.RegisterGradient("GuidedRelu")
def _GuidedReluGrad(op, grad):
    return tf.select(0. < grad, gen_nn_ops._relu_grad(grad, op.outputs[0]), tf.zeros(grad.get_shape()[1:]))


def target_category_loss(x, category_index, nb_classes):
    return tf.multiply(x, K.one_hot([category_index], nb_classes))


def target_category_loss_output_shape(input_shape):
    return input_shape


def normalize(x):
    # utility function to normalize a tensor by its L2 norm
    return x / (K.sqrt(K.mean(K.square(x))) + 1e-5)

def backend_reshape(x):
    shape = (my_batch_size, img_width, img_height, 3)
    return K.reshape(x, shape)

def reproduce(x):
    return x

tuned_vgg_weights_path = './tuned_vgg_model.h5'
ldir = '/N/u/zehzhang/cogsci16/valid/0/'
name = '01_20150530_16859_sync_frames_parent_img_06600.jpg'
img_path = ldir + name
img_width = 224
img_height = 224
my_batch_size = 1


nb_classes = 24
category_index = 0
layer_name = 'block5_conv3'


with tf.get_default_graph().gradient_override_map({'Relu': 'GuidedRelu'}):
    model = Sequential()
    base_model = VGG16(weights='imagenet', include_top=False, input_shape=(img_width, img_height, 3))
    model.add(base_model)
    model.add(Flatten())
    model.add(Dense(4096, activation='relu'))
    model.add(Dense(4096, activation='relu'))
    model.add(Dense(24, activation='relu'))
    model.add(Activation('softmax'))
    model.load_weights(tuned_vgg_weights_path)
    print(model.summary())



conv_output = [l for l in model.layers[-6].layers if l.name is layer_name][0].output
grads = K.gradients(model.layers[-2].output[0, 0], model.layers[-6].layers[-2].output)[0]

img = load_img(img_path, target_size=(img_width, img_height))
ori_w, ori_h = load_img(img_path).size

img_array = img_to_array(img)
gradient_function = K.function([model.layers[0].input], [conv_output, grads])

output, grads_val = gradient_function([preprocess_input(img_array[np.newaxis, :])])

[jubjamie]

Would you mind editing your above comment to put it all in code with correct indentation and formatting please? Just to make sure everything is written as it should be! In the mean time I will try a little test here.

[zehzhang]

@jubjamie
Yes wait a minute. In fact I'm now struggling to add the indentation.

[zehzhang]

@jubjamie
I have upated my code above with correct indentation.

[jubjamie]

Problem solved. Can you please confirm what version of tensorflow you are using?

[zehzhang]

@jubjamie
Thanks!
I'm using the most recent version and I guess that's the reason because I found in the new version of tensorflow, there was one file named select.py added to tensorflow.contrib.graph_editor and it seems tensorflow.contrib is lazily imported in init.py.

Am I right? If it's the reason, except using the previous version, is there any way I can solve it?

[jubjamie]

tf.select is deprecated that's why. I can't believe I didn't spot that at first!
In TF1.0 tf.select() is not an op in the control flow group. Can I suggest that you use tf.where() and see if you get the desired results. In my brief testing you should get identical functionality.

Let me know if that solves your problem.

[zehzhang]

@jubjamie
After running python -c 'import tensorflow as tf; print(tf.version)', I get:
1.0.1

So the exact version should be 1.0.1 I think.

[jubjamie]

Yeah that's fine. Try using tf.where() instead as tf.select is a deprecated control function. Maybe that's why you are seeing it in contrib? Would have to ask a Tensorflower!
Does this answer your issue?

[zehzhang]

@jubjamie
Yes and thanks a lot!
It seems in the latest version, tf.select is removed and tf.where is a substitute!

[jubjamie]

Yes. tf.where used to have a similar but different functionality. If this solved then feel free to close the issue.
Hope I could help!

[zehzhang]

@jubjamie
Could I ask one last-minute question?
I try to call tf.zeros(grad.gets_shape()), but the shape of my grad is (?, 24) and thus it fails to create a zero tensor because tf.zeros requires a specific shape. Is there any way I can create the zero tensor I want?

[jubjamie]

I'm out the office now but if you can create a stackoverflow question, link it here and then close the issue that would be great as Google like to keep this clear for bugs. I'll try and get back to you tomorrow.

[skye]

@jubjamie thank you for your responses! I'm gonna close this issue.