Missing Trainable Variables and Variables

[YingzhouLi]

System information

• Have I written custom code (as opposed to using a stock
  example script provided in TensorFlow): Yes
• OS Platform and Distribution (e.g.,
  Linux Ubuntu 16.04): Ubuntu 19.04
• TensorFlow installed from (source or
  binary): TF 2.1 installed from pip
• Python version: Python 3.7.5

import os
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
import tensorflow as tf

class FooLayer(tf.keras.layers.Layer):
    def __init__(self, siz):
        super(FooLayer, self).__init__()
        self.siz = siz
        self.buildFoo(siz)

    def call(self, in_data):
        Foo0 = tf.multiply(in_data,self.FooTns0)
        FooList = []
        FooList.append(Foo0)
        for it in range(1,self.siz+1):
            tmp = tf.multiply(FooList[it-1],self.FooTns[it-1])
            FooList.append(tmp)
        return FooList[self.siz]

    def buildFoo(self,siz):
        self.FooTns0 = tf.Variable(1, name="TNS0")
        self.FooTns = []
        for it in range(0,self.siz):
            self.FooTns.append(tf.Variable(it, name="TNS"+str(it+1)))

class FooModel(tf.keras.Model):
    def __init__(self, siz):
        super(FooModel, self).__init__()
        self.flayer = FooLayer(siz)

    def call(self, in_data):
        return self.flayer(in_data)

model = FooModel(5)

for v in model.trainable_variables:
    print(v.name)

for v in model.variables:
    print(v.name)

The output currently is only:

TNS0:0
TNS0:0

While the expected output is listing all 6 tensors, ''self.FooTns0'' and ''self.FooTns''.

[Saduf2019]

i am able to replicate this issue, please find the gist here

[YingzhouLi]

Thank you @Saduf2019 .
@jvishnuvardhan , could you please let me know whether this is a bug in tensorflow 2? Is there an easy way to work around this python array? In my complicated code, the variables are of different sizes and I cannot simply concatenate them as a bigger tensor.

[gulshanrana10]

@YingzhouLi, As you haven't called for the call function, thatswhy it isn't being concatenated and is not showing all 6 vectors. Probably certain changes might help in showing all 6 vectors and finally call function will concatenate all of the vectors-
Please find the changes here-colab_link

[YingzhouLi]

@gulshanrana10 Thank you very much for the fix. It works. However, I am very much confused by the mechanism behind this. I am relatively new to python but familiar with C++ class. Why tensorflow cannot catch self.FooTns directly, which is also a member of the class as self.array in init?

[qlzh727]

Thanks for reporting the issue, it seems to be a bug in layer weight tracking logic. There are a few walkaround at the moment when we fix the issue:

1. Change the base class to tf.keras.Model, which currently track the weights correclty.
2. Use layer.add_weights() and append the result into your list, which also will do the correct thing.

[YingzhouLi]

Thank you. It is good to know that the issue is not caused by some unknown nature of python. The workaround works for me and look forward to the update.

[sachinprasadhs]

Was able to reproduce your issue in Tf Nightly 2.6.0-dev20210525, please find the gist here. Thanks!

[chunduriv]

Was able to reproduce your issue in TFv2.7 and Tf Nightly(2.8.0-dev20211125), please find the gist here. Thanks!

[tensorflowbutler]

Hi There,

This is a stale issue. As you are using an older version of tensorflow, we are checking to see if you still need help on this issue. Please test the issue with the latest TensorFlow (TF2.7 and tf-nightly). If the issue still persists with the newer versions of TF, please feel free to open it in keras-team/keras repository by providing details about the issue and a standalone code to reproduce the issue. Thanks!

Please note that Keras development has moved to a separate Keras-team/keras repository to focus entirely on only Keras. Thanks!

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No