Keras generators does not work

[robertanto]

Dear kymatio team,

I am trying to train a Keras Neural Network based on the Scattering1D layer. Due to the huge quantity of data, I have implemented a keras.utils.Sequence generator for accessing to my data. But I got the following error:

Train for 10 steps
 1/10 [==>...........................] - ETA: 0s

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-5-ef3a40a87ee6> in <module>()
      5 )
      6 
----> 7 model.fit(DataGenerator())
 ----------------------------- STACK --------------------------------------
/usr/local/lib/python3.7/dist-packages/kymatio/scattering1d/backend/tensorflow_backend.py in pad_1d(x, pad_left, pad_right, mode, value)
     62         The tensor passed along the third dimension.
     63     """
---> 64     if (pad_left >= x.shape[-1]) or (pad_right >= x.shape[-1]):
     65         if mode == 'reflect':
     66             raise ValueError('Indefinite padding size (larger than tensor).')

TypeError: '>=' not supported between instances of 'int' and 'NoneType'

I am using Tensorflow 2.0.0, in particular, tensorflow.keras together with kymatio-0.2.1.

Here a simple code to reproduce the error (I tried on google colab, if you try it on colab remember to downgrade tensorflow to 2.0.0).

import numpy as np
import tensorflow as tf
from kymatio.keras import Scattering1D
from tensorflow import keras
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input,Dense, Flatten
from tensorflow.keras.utils import Sequence

class DataGenerator(Sequence):

    def __len__(self):
        'Denotes the number of batches per epoch'
        return 10

    def __getitem__(self, index):
        'Generate one batch of data'
        X = np.random.rand(1000,3200)
        y = np.random.rand(1000,1024)
        return X, y

# Params
J=9
Q=8

# Model
inputs = Input(shape=(3200,))
  
x = Scattering1D(J,Q)(inputs)
x = Flatten()(x)
y = Dense(1024,'softmax')(x)

# Final Model
model = Model(inputs=inputs,outputs = y)

model.compile(
    loss=keras.losses.CategoricalCrossentropy(),
    optimizer=keras.optimizers.RMSprop(),
    metrics=["accuracy"],
)

model.fit(DataGenerator())

[OverLordGoldDragon]

Is this based on any Kymatio example? I'd try eliminating any Nones from tensor shapes as that's the source of error.

[robertanto]

The error has been generated by executing the reported code.

The code is not based on any kymatio example because they assume that the dataset fits the main memory, but it is not true in most of the cases.

[kkm000]

---> 64     if (pad_left >= x.shape[-1]) or (pad_right >= x.shape[-1]):

TF2 Dataset and Sequence are executed in graph mode, i.e. in a @tf.function. I think that the right function to use here is tf.shape, but with TF/Keras it's always kinda hit and miss...

@robertanto: This is almost 2 years old, but just in case, have you found a workaround? I don't think that a downgrade to 2.0 is still necessary. Does Dataset cause the same issue? I guess it should.

@OverLordGoldDragon:

I'd try eliminating any Nones from tensor shapes

This isn't really a preferred approach, generally speaking. None traces a function with a wildcard dimension, handled at runtime; an explicitly specified dimension causes retracing when it changes. This takes a long time, and the extra compiled function graphs consume memory; also, graphs with multiple instances of a function may be stored within the model, bloating the file (that's an option to all Save variants that work on graphs). Explicit values are ok only for known static dimensions.

[robertanto]

Unfortunately, I am not working on that project anymore. The only workaround I found was to precompute the features and then read them as images.