TypeError while training theStable diffusion model.

[sayannath]

Working on Fine-tuning Stable Diffusion

Current Behavior:

epochs = 1
ckpt_path = "finetuned_stable_diffusion.h5"
ckpt_callback = tf.keras.callbacks.ModelCheckpoint(
    ckpt_path,
    save_weights_only=True,
    monitor="loss",
    mode="min",
)
diffusion_ft_trainer.fit(training_dataset, epochs=epochs, callbacks=[ckpt_callback])

print(training_dataset)

<_PrefetchDataset element_spec={'images': TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name=None), 'tokens': TensorSpec(shape=(None, 77), dtype=tf.float64, name=None), 'encoded_text': TensorSpec(shape=(None, 77, 768), dtype=tf.float32, name=None)}>

Error:

[divyashreepathihalli]

@sayannath are you facing this error while running the example colab as is without making any changes?
Tagging the authors of this example - @sayakpaul could you please take a look?

[sayakpaul]

I think it might be because of the version changes. I would suggest checking the versions around the time this tutorial was published and seeing if that works.

[sayannath]

I think it might be because of the version changes. I would suggest checking the versions around the time this tutorial was published and seeing if that works.

Yes, I have tried using that exact version but it was throwing errors while importing the ClipTokenizer and the other modules.

Here is the issue I created. Link
The solution Keras team provided. Link

[divyashreepathihalli]

@sayannath this should be fixed now, can you check?