ValueError: Graph disconnected: cannot obtain value for tensor Tensor…The following previous layers were accessed without issue:

[boluoyu]

I want to obtain the output of intermediate sub-model layers with tf2.keras.Here is a model composed of two sub-modules:

    input_shape = (100, 100, 3)

    def model1():
        input = tf.keras.layers.Input(input_shape)
        cov = tf.keras.layers.Conv2D(filters=32, kernel_size=3, strides=1,name='cov1')(input)
        embedding_model = tf.keras.Model(input,cov,name='model1')
        return embedding_model

    def model2(embedding_model):

        input_sequence = tf.keras.layers.Input((None,) + input_shape)

        sequence_embedding = tf.keras.layers.TimeDistributed(embedding_model,name='time_dis1')

        emb = sequence_embedding(input_sequence)
        att = tf.keras.layers.Attention()([emb,emb])
        dense1 = tf.keras.layers.Dense(64,name='dense1')(att)
        outputs = tf.keras.layers.Softmax()(dense1)

        final_model = tf.keras.Model(inputs=input_sequence, outputs=outputs,name='model2')
        return final_model

    embedding_model = model1()

    model2 = model2(embedding_model)
    print(model2.summary())

output:

Model: "model2"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_2 (InputLayer)            [(None, None, 100, 1 0                                            
__________________________________________________________________________________________________
time_dis1 (TimeDistributed)     (None, None, 98, 98, 896         input_2[0][0]                    
__________________________________________________________________________________________________
attention (Attention)           (None, None, 98, 98, 0           time_dis1[0][0]                  
                                                                 time_dis1[0][0]                  
__________________________________________________________________________________________________
dense1 (Dense)                  (None, None, 98, 98, 2112        attention[0][0]                  
__________________________________________________________________________________________________
softmax (Softmax)               (None, None, 98, 98, 0           dense1[0][0]                     
==================================================================================================
Total params: 3,008
Trainable params: 3,008
Non-trainable params: 0

and then,I want to get output intermediate layer of model1 and model2:

    model1_output_layer = model2.get_layer('time_dis1').layer.get_layer('cov1')
    output1 = model1_output_layer.get_output_at(0)
    output2 = model2.get_layer('dense1').get_output_at(0)

    output_tensors = [output1,output2]
    model2_input = model2.input
    submodel = tf.keras.Model([model2_input],output_tensors)
    input_data2 = np.zeros((1,10,100,100,3))

    result = submodel.predict([input_data2])
    print(result)

Running in tf2.3 ,the error I am getting is:

 File "/Users/bouluoyu/anaconda/envs/tf2/lib/python3.6/site-packages/tensorflow/python/keras/engine/functional.py", line 115, in __init__
    self._init_graph_network(inputs, outputs)
  File "/Users/bouluoyu/anaconda/envs/tf2/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 457, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "/Users/bouluoyu/anaconda/envs/tf2/lib/python3.6/site-packages/tensorflow/python/keras/engine/functional.py", line 191, in _init_graph_network
    self.inputs, self.outputs)
  File "/Users/bouluoyu/anaconda/envs/tf2/lib/python3.6/site-packages/tensorflow/python/keras/engine/functional.py", line 931, in _map_graph_network
    str(layers_with_complete_input))
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(None, 100, 100, 3), dtype=float32) at layer "cov1". The following previous layers were accessed without issue: ['time_dis1', 'attention', 'dense1']

But the following code works:

    model1_input = embedding_model.input
    model2_input = model2.input

    submodel = tf.keras.Model([model1_input,model2_input],output_tensors)

    input_data1 = np.zeros((1,100,100,3))
    input_data2 = np.zeros((1,10,100,100,3))

    result = submodel.predict([input_data1,input_data2])
    print(result

But not what I want.This is strange, model1 is part of model2, so why do we need to input an extra tensor input_data1 ? Sometimes,it is hard to get an extra tensor,especially for complex models.What should I do? Do we need a new API to support this functionality?

[amahendrakar]

Was able to reproduce the issue with TF v2.3 and TF-nightly. Please find the gist of it here. Thanks!

[boluoyu]

any updates ?

[gowthamkpr]

@boluoyu Looks like this is an implementation issue. Please take a look at this issue here and let me know if it helps. Thanks!

[boluoyu]

@gowthamkpr No,This issue is very different from that one. These are two completely unrelated issue.

    embedding_model = model1()
    model2 = model2(embedding_model)
    print(model2.summary())

    input_data2 = np.zeros((1,10,100,100,3))

    result = model2.predict(input_data2)
    print(result) #works

Do you see it ? when I use the raw model2 to predict ,JUST input one tensor ,no problem.But I MUST input two tensor when I want to get output intermediate layer of model1 and model2.

    output1 = model1_output_layer.get_output_at(0)
    output2 = model2.get_layer('dense1').get_output_at(0)

    output_tensors = [output1,output2]
    isubmodel = tf.keras.Model([model1_input,model2_input],output_tensors) #input two tensor

    input_data1 = np.zeros((1,100,100,3))
    input_data2 = np.zeros((1,10,100,100,3))

    result = submodel.predict([input_data1,input_data2])

Model1 is part of model2, we don't need an extra tensor input_data1,I hope the new API changes it.

[arvoelke]

I have encountered this same issue as well. In general, it happens any time you use some functional sub-model as a layer in your model, and then you want to use a tensor from that sub-model in the outer model (e.g., exposing it as an extra output to debug or analyze a model).

Example:

import tensorflow as tf

inp1 = tf.keras.Input(1)
h1 = tf.keras.layers.Dense(1)(inp1)
out1 = tf.keras.layers.Dense(1)(h1)
model1 = tf.keras.Model(inputs=inp1, outputs=out1)

inp2 = tf.keras.Input(1)
h2 = model1(inp2)
out2 = tf.keras.layers.Dense(1)(h2)
model2 = tf.keras.Model(inputs=inp2, outputs=[out2, model1.layers[1].output])

=>

ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(None, 1), dtype=float32) at layer "dense". The following previous layers were accessed without issue: ['functional_1']

The issue does not occur if the submodel is at the same "level" as the outer model, or if you explicitly make the required tensor an output of the sub-model. It's not clear to me if there are any other ways around this, but this seems to be the intent with the functional model API.

[michaelpulis]

I am also experiencing this issue. I wish to perform feature extraction on my siamese network model (which contains two sub-models) and I am facing this issue.

[Yannik1337]

As a workaround, you can add the embedding layers to the model's outputs:

model = tf.keras.Model(inputs=[A, B], outputs=[distance, embeddingA, embeddingB])

where A and B are the inputs to the sub-networks, and embeddingA and embeddingB are the outputs (the embeddings) of these sub-networks. Not beautiful, but it works in a custom train loop.

[sanatmpa1]

@boluoyu,

Can you take a look at the above workaround proposed by @Yannik1337 and let us know if it helps? Thanks!

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No