"<tensor> is not an element of this graph." when loading model.

[ghost]

Update:

I am using the model which I am loading below just to generate features for another model. Could it be that the problem is because I am trying to load multiple models in one session?

---

When I load the model in word2vec_25_tar-category.h5.zip using load_model in one of my modules I am getting

...
control_dependencies
    c = self.as_graph_element(c)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2318, in as_graph_element
    return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2397, in _as_graph_element_locked
    raise ValueError("Tensor %s is not an element of this graph." % obj)
ValueError: Tensor Tensor("Softmax:0", shape=(?, 13), dtype=float32) is not an element of this graph.

Traceback (most recent call last):
  File "/media/Data/workspaces/git/master-thesis/python/thesis/absa/slot1/constrained/ffn.py", line 382, in <module>
    verbose=True)
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 1532, in fit_generator
    str(generator_output))
ValueError: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None

However, I do not get this in all modules.

This is how I load it:

load_model(model_path,
  custom_objects={
    'dense_sum': dense_sum,
    'merge_average': merge_average
  })

These are the custom objects:

def dense_sum(x):
    from keras import backend as K
    return K.sum(x)

def merge_average(x):
    from keras import backend as K
    return x[0]/K.sum(x[1])

Why can I use the model in one module but not in another? I am pretty helpless because I don't see what the issue is. I definitely load the same file.

What's wrong here?

[piraka9011]

I'm having a similar issue (I think) but it occurs when I call the predict() method on a model. I am given a raise ValueError...is not an element of this graph.

Any suggestions?

[ghost]

Change the backend to Theano. change it back when the issue is resolved.

[darespaco]

Hi silentsnooc,

I'm facing a similar issue. I also need to load a keras model from an h5 file at the same time I generate a tf seq2seq model. When I load the models independently it works well, the problem appears when I try to integrate those two actions. Did you finally get to solve this issue?

Thanks in advance

[joaospinto]

I had a problem similar to that of @piraka9011 which was solved by calling model._make_predict_function() right after loading the trained model. If this does not work, check out #2397.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[frankcarey]

Just got this error all of a sudden (Was working fine earlier today).. model._make_predict_function() worked for me as well to solve it, but weird!

[avelkoski]

Calling model._make_predict_function() worked for me.

[anujgupta82]

This worked for me:

def load_model():
	global model
	model = ResNet50(weights="imagenet")
        # this is key : save the graph after loading the model
	global graph
	graph = tf.get_default_graph()

While predicting, use the same graph

    with graph.as_default():
	preds = model.predict(image)
	#... etc

[richinhim]

i had the same environment. env: flask + tensorflow + keras (dev env: windows 10 pro)
error log is

ValueError: Tensor Tensor("avg_pool/AvgPool:0", shape=(?, 1, 1, 2048), dtype=float32) is not an element of this graph.

this worked for me:

global graph
graph = tf.get_default_graph()

while predicting, use the same graph
with graph.as_default():
feature = resnet_vgg.predict(npImg)

thanks to anujgupta82

[minus31]

I Solved! I made a function which runs without app.route(), the decorator. And I loaded model in the function. And then I called the function when I need prediction

[hachreak]

Same issue!
model._make_predict_function() resolve the problem but the fix looks weird... XD
Somebody know why this happen? There is a context that get lost?

[offchan42]

@minus31 I'm confused. What did you mean? You load model every time you predict?
Also, model._make_predict_function() work with my model too. My model is not official and is trained from scratch.
So it seems like this is the way to go for now. Is there an official way to fix this yet?

PS. I use tensorflow.keras module, not keras directly.

[wqp89324]

In my case, I have a flask endpoint that passes a key to a process_algorithm method.
The process_algorithm method will each time load a pre-trained keras model and then make a prediction. It seems that the first time the process_algorithm method is called, it works fine, but the next time, there will be this error:
Cannot interpret feed_dict key as Tensor: Tensor Tensor("Placeholder:0", shape=(196, 128), dtype=float32) is not an element of this graph
at load_model(). Any way to solve this?

[SaratM34]

changing backend to theano worked for me

[Kobzol]

@wqp89324 try calling keras.backend.clear_session() on each request

[hgasimov]

@wqp89324 I had the same issue. Calling keras.backend.clear_session() each time before loading a model solved the problem.

[josvaler]

model._make_predict_function() which works perfectly within python 3.6.
Backend

import tensorflow as tf
print(tf.version)
1.12.0
import keras as ke
Using TensorFlow backend.
print(ke.version)
2.2.4

[yashmehta14]

This worked for me:

def load_model():
	global model
	model = ResNet50(weights="imagenet")
        # this is key : save the graph after loading the model
	global graph
	graph = tf.get_default_graph()

While predicting, use the same graph

    with graph.as_default():
	preds = model.predict(image)
	#... etc

I am also doing the same thing shown by you but then also i am etting the same error...First time i will get the output but when i am trying to click the link for second time it is giving the error.. Can u help me please ?

[arkitpatel]

I faced the same issue when I was passing an instance of loaded model to another thread which was predicting. I changed my execution and made the model load in the same thread as the one which was predicting. I am not sure if this solves anyone's problem but hope so it helps someone.

[yashmehta14]

from flask import Flask, request
import os, cv2, keras, random
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
from keras.models import Model
from keras.optimizers import Adam
from sklearn.utils import shuffle
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten, BatchNormalization
from keras.layers import Conv2D, MaxPooling2D
from keras.utils import np_utils, to_categorical
from keras.models import load_model
import tensorflow as tf
global graph, graph1, model1, model2
app = Flask(name)

@app.route("/")
def hello():
filepath = request.args.get('filepath')
folder = 'data'+ str(random.randint(1,1000))
returnstring ='not executed'
os.mkdir(folder)
print(filepath)
actualpath = filepath
vidcap = cv2.VideoCapture(actualpath)
count = 0
while True:
success,image = vidcap.read()
if not success:
break
cv2.imwrite(os.path.join(folder,"frame{:d}.jpg".format(count)), image) # save frame as JPEG file
count += 1

model1 = Sequential()

model1.add(Conv2D(32, 3, padding='same', input_shape=(256,256,3), activation='relu'))
model1.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_first"))

#print("First layer...")
model1.add(Conv2D(32, 3, padding='same', activation='relu'))
model1.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_first"))

#print("Second layer...")
model1.add(Conv2D(64, 3, padding='same', activation='relu'))
model1.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_first"))

model1.add(Flatten()) 
model1.add(Dense(64))
model1.add(Activation('relu'))
model1.add(Dropout(0.5))
model1.add(Dense(1))
model1.add(Activation('sigmoid'))

model1.load_weights("Weights_model1.h5")
graph1 = tf.get_default_graph()

model2 = Sequential()

model2.add(Conv2D(32, 3, padding='same', input_shape=(256,256,3), activation='relu'))
model2.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_first"))

#print("First layer...")
model2.add(Conv2D(32, 3, padding='same', activation='relu'))
model2.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_first"))

#print("Second layer...")
model2.add(Conv2D(64, 3, padding='same', activation='relu'))
model2.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_first"))

model2.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors
model2.add(Dense(64))
model2.add(Activation('relu'))
model2.add(Dropout(0.5))
model2.add(Dense(1))
model2.add(Activation('sigmoid'))

model2.load_weights("Weights_model2.h5")
graph2 = tf.get_default_graph()

Training_water_data = 'C:\\Users\\ysmehta\\'+folder+'\\'
training_water_data = [x for x in sorted(os.listdir(Training_water_data))]
x_train_water_data = np.empty((len(training_water_data),256,256,3),dtype = 'float32')
for i,name  in enumerate(training_water_data):
    im = cv2.imread(Training_water_data + name).astype('int16').astype('float32')/255.
    im = im[:,:,:]
    im = cv2.resize(im,dsize = (256,256),interpolation = cv2.INTER_NEAREST)
    x_train_water_data[i] = im

x_train_water_data = x_train_water_data.reshape(x_train_water_data.shape[0],256,256,3)

for i in range(0,x_train_water_data.shape[0]):
    A = x_train_water_data[i]
    B = A.reshape(1,256,256,3)
    with graph1.as_default():
        prediction1 = model1.predict(B,verbose=0)
return returnstring;

if name == "main":
app.run(host='131.163.144.196',port=5000)

This is the code... Can u help me ?

[Suji04]

@yashmehta14 I was having a similar problem. I fixed it by adding a function which can be called later

def get_model():
    global model
    model = load_model("model_new.h5")
    model._make_predict_function()
    print("Model Loaded")

[ivsanro1]

@joaospinto hero!

[tomer-ben-david]

I had a problem similar to that of @piraka9011 which was solved by calling model._make_predict_function() right after loading the trained model. If this does not work, check out #2397.

this has fixed it for me as well what does it do?? :)

[DhruvMevada]

I had a problem similar to that of @piraka9011 which was solved by calling model._make_predict_function() right after loading the trained model. If this does not work, check out #2397.
@joaospinto It works for me. just do model._make_predict_function()

[summerswu]

ran into the same problem when trying to deploy a CNN to production as a flask app, to summarize, there seem to be two fixes that worked with me

1. Switch to a Theanos backend
2. Add the line model._make_predict_function() right after you have loaded your model

[irwan15]

\\Pada tanggal 4 Jul 2019 07:23, Summers Wu <notifications@github.com> menulis:ran into the same problem when trying to deploy a CNN to production as a flask app, to summarize, there seem to be two fixes that worked with me Switch to a Theanos backendAdd the line model._make_predict_function() right after you have loaded your model —You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or mute the thread.

[muhkhoi]

@yashmehta14 I was having a similar problem. I fixed it by adding a function which can be called later

def get_model():
    global model
    model = load_model("model_new.h5")
    model._make_predict_function()
    print("Model Loaded")

thanks sir it works for me after struggling whole day looking for an answer

[Mrjaggu]

I had the same issue while using flask + keras
global graph
graph = tf.get_default_graph()

with graph.as_default():
res = model.predict()

this worked for me
Thanks, @anujgupta82

[GerardWalsh]

I had the same problem when using ResNet50. Solved the same as @Mrjaggu, with input from @anujgupta82:

def load_model(): global model model = ResNet50(weights="imagenet") global graph graph = tf.get_default_graph()
and using
with graph.as_default(): preds = model.predict(image)
in my predict function.

[kesmith21]

Thanks to @Mrjaggu , @anujgupta82 , @GerardWalsh and others!
I spent at least a day trying to find how to correct the error.
Here's the final result:
def get_model():
global graph
graph = tf.get_default_graph()
with graph.as_default():
global model
model = load_model('model_WheelPrediction.h5')
Then in the app.route:
@app.route("/predict", methods=["POST"])
def predict():
with graph.as_default():
message = request.get_json(force=True)
encoded = message['image']
decoded = base64.b64decode(encoded)
image = Image.open(io.BytesIO(decoded))
processed_image = preprocess_image(image, target_size=(224, 224))
prediction = model.predict(processed_image).tolist()

[harshitsinghai77]

This works for me
global graph
graph = tf.get_default_graph()
def nsfw_predict():
with graph.as_default():
predict = model.predict('nsfw/uploads/image_nsfw.jpg')
return predict

[gustavz]

using tf.get_default_graph() and graph.as_default() does not solve my problem: still seeing the following error when trying to run keras models on flask apps:

tensorflow.python.framework.errors_impl.FailedPreconditionError: Error while reading resource variable dense_3/bias from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource localhost/dense_3/bias/N10tensorflow3VarE does not exist.
         [[{{node dense_3/BiasAdd/ReadVariableOp}}]]

[GaoQ1]

@gustavz I have the same problem, could you solve it?

[lukkascost]

Works with clean session before the model load and predict.

from keras import backend as K

@api.expect()
    def post(self):
        K.clear_session()
        loaded_model_f32 = load_model("app/Utils/model.h5")

        file = request.files['file']
        file.save("tmp.wav")
        filestr = request.files['file'].read()
        data = wav.read('tmp.wav')[1]

        input_data = np.zeros((1, 50213))
        input_data[0][:size] = data[:size]

        predicted = loaded_model_f32.predict(input_data)

        result = {"result":
            {
                "time_to_load_model": time_load_model,
                "time_to_predict": time_predict_pattern,
                "classe": CLASSES[np.argmax(predicted)]
            }
        }
        return result

[secsilm]

If it works within REPL but not flask, try running the flask app with gunicorn and set mode to sync. Mode gevent will get the error.

[maxwellmandela]

I had a problem similar to that of @piraka9011 which was solved by calling model._make_predict_function() right after loading the trained model. If this does not work, check out #2397.

works perfect

[rajashree921]

I Solved! I made a function which runs without app.route(), the decorator. And I loaded model in the function. And then I called the function when I need prediction

thank you! Your suggestion was a godsend. I had no idea what was causing the error

[shahdghorsi]

I had a problem similar to that of @piraka9011 which was solved by calling model._make_predict_function() right after loading the trained model. If this does not work, check out #2397.

Hello, thank you for the answer but, I am facing this error when using this function :
TypeError: _make_predict_function() takes 1 positional argument but 2 were given
Even though I am not passing 2 arguments I am just passing an image as a dims expanded numpy array. Do you have any suggestions?

[Spidy20]

Guys, if you are getting this error in flask environment. the solution is simple, create a function for a prediction and load a model in that function instead of declaring it globally. It will solve your error.
Thanks

[shivasuri]

@Spidy20 I don't think that is a good solution because it will require the model to be reloaded every time the predict API is called.

For the record, I ran into an issue with model._make_predict_function() and was able to solve it following this stackoverflow post.

[YatinChachra]

Solve it by:

Instead of :
app.run(debug=False)

Use:

if name == 'main':
app.run(debug=False, threaded=False)

[Spidy20]

@YatinChachra Thanks for it, Actually it is the perfect solution. Yes, you are right @shivasuri my solution was temporary to solve the error. But this solution is perfect. Thanks for correcting me:)

[smb-maker]

I was having the same problem like piraka9011 it was making error when predict function was called. but it works fine for me when i use piraka9011 single line of code
model._make_predict_function()
just after loading the model for prediction..

[gg22mm]

It's solved
1、vi /root/.keras/keras.json replace "tensorflow" to "theano"
2、KERAS_BACKEND=theano python -c "from keras import backend;"

it right ! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

but too slow ~

time: 38.32983899116516

[webwarz]

The tensorflow version is tf1.X
you should to ensure load_model and predict_model code using the same graph
so , after load_model ,you should :

model_graph = tf.get_defalut_graph() global model_graph #(if load_model and predict_model is not in one code)

and then ,predicet_model ,you can :

from load_model(the load_model module) import model_graph
with model_graph as_defalut():
predict_model(code for predict)