This model was created with a U-Net model trained on a fashion MNIST dataset. Then, the model was extended with various functionalities including forward and reverse diffusion, group normalization, GELU activation function, rearrange pooling, sinusoidal position embeddings, classifier-free diffusion guidance (CFDG), and weighted reverse diffusion.
A Contrastive Language-Image Pre-Training (CLIP) model was also used to generate images from text and enhance the diffusion model with the CLIP encodings.
Finally, I used the diffusion model to generate images from the MNIST dataset which is similar to the fashion MNIST dataset and used a classifier enhanced with CFDG which was already trained on the MNIST to identify the images, obtaining 98% accuracy. Other performance metrics could be used but this is a start.
Screenshots:
