tf.keras.callbacks.Tensorboard: write_images does not visualize Conv2D weights

[maxstrobel]

I moved this issue from tensorflow/tensorflow#28767

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
• TensorFlow version (use command below): v1.12.2-0-gcf74798993 1.12.2
• Python version: 3.6.5

Describe the current behavior
When I want to have a look at the weights of Conv2D filters in TensorBoard, only their biases get logged (see attached image). I looked for the corresponding source code and found the following snippet:
https://github.com/tensorflow/tensorflow/blob/6612da89516247503f03ef76e974b51a434fb52e/tensorflow/python/keras/callbacks.py#L951-L983
The problem seems to be that Conv2D weights have a 4d shape [H_kernel, W_kernel, C_in, C_out], which is not intended as convolutional layers case in the above code.

Describe the expected behavior
I would expect that the convolutional weights are visualized. I know this would be a huge amount of images (C_in * C_out), but I think the current behaviour is confusing.

Code to reproduce the issue

import tensorflow as tf

cifar10 = tf.keras.datasets.cifar10

(x_train, y_train), (x_test, y_test) = cifar10.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
print(type(x_train))
model = tf.keras.models.Sequential([
    tf.keras.layers.InputLayer(input_shape=(32, 32, 3)),
    tf.keras.layers.Conv2D(filters=16, kernel_size=3, padding='same', activation='relu'),
    tf.keras.layers.Conv2D(filters=32, kernel_size=3, padding='same', activation='relu'),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])
model.summary()

tensorboard = tf.keras.callbacks.TensorBoard(log_dir=f"../../../logs", histogram_freq=1,
                                             write_images=True, write_grads=True)
csvlogger = tf.keras.callbacks.CSVLogger('train.log')
model.compile(optimizer=tf.keras.optimizers.Adam(lr=1e-4),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5, callbacks=[tensorboard, csvlogger], validation_data=(x_test, y_test))

Other info / logs

[menon92]

@maxstrobel I'm facing same issue. Do you solve it ?

[maxstrobel]

Hi @menon92,

No I did not dive deeper into it. However, the issue seems to be also present in TF2.0.
If I find time and look deeper into this, I will share my findings.

[nfelt]

cc @caisq FYI re tensor visualization

[rafwaf]

@rmothukuru , @nfelt , @caisq , I have the same issue. Tensorboard will not visualize Kernels of Conv2D layers, only biases.

[MZehren]

I have a related issue: On my side, only the first 3 Kernels of the first Conv2D layer are displayed. Is it somewhat related to the signature of the tf.summary.image function which has max_outputs=3 by default?

tf.summary.image(
    name,
    data,
    step=None,
    max_outputs=3,
    description=None
)

[wchargin]

@MZehren: Yes: as you note, we only write the first max_outputs images
passed to any single call to tf.summary.image (to save time on PNG
encoding and space on disk). The TensorBoard Keras callback calls
tf.summary.image without overriding the default for max_outputs,
so there’s no way to visualize more than the first 3 kernels via the
callback at this time.

[rafwaf]

Hi @wchargin , I wonder if in the meantime, you might be able to answer my following question, while we "stat:awaiting tensorflower"... Why is there a label: "type: feature"? Isn't this a bug? Isn't Tensorboard (and it's callback) meant to currently be able visualize the Kernels of Conv2D layers already?
Thanks in advance.