Skip to content

ClassyPics - Class-Conditional Image Generation: Generate images of specific objects using diffusion. Train a conditional stable diffusion model from scratch and develop an interface to input class labels and produce corresponding images.

Notifications You must be signed in to change notification settings

its-ishan/ClassyPics

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ClassyPics - Class-Conditional Image Generation

Image generation is a cutting-edge application of deep learning, especially beneficial when training data is limited. In this hackathon challenge, participants will tackle the task of generating images of specific objects using conditional stable diffusion models. Task: Participants are tasked with designing and implementing a conditional stable diffusion model capable of generating synthetic data samples corresponding to specific class labels. The main objectives include:

  1. Training the model effectively.
  2. Developing a user interface that accepts class labels and produces corresponding images. Prerequisites:
  3. Solid understanding of Deep Learning Fundamentals.
  4. Proficiency in Python and Deep Learning Frameworks.
  5. Knowledge of Probability and Statistics.
  6. Experience with Image Processing and Computer Vision. Considerations: Participants must implement mechanisms for class-conditional generation, enabling the model to produce synthetic data samples tailored to specific class labels. Datasets: Two different class-labelled datasets will be provided to the participants. Evaluation:
  7. Quality of Generated Images: Evaluation will involve a subjective assessment of the quality of the generated images, considering factors like clarity, realism, and relevance to the specified class label.
  8. Additional Evaluation Metrics: Detailed evaluation metrics will be shared soon to provide participants with clear guidelines for assessing the performance of their models.
  9. Efficiency of User Interface: The efficiency and usability of the user interface developed for generating images will be evaluated, focusing on factors such as responsiveness, ease of use, and overall user experience.

The Root folder is CP2, inside it, main.py contains the main program.

Directroy Structure.

  1. config - contains all the configuration files.
  2. data - contains the datasets on which the models are trained
  3. dataset - contains classes that can load the data, mainly animal_dataset.py and mnist_dataset.py
  4. evaluation - contains code for calculating the metrics (inception score and FID score)
  5. latents - dummy directory
  6. mnist - it contains another directory naming cond_class_samples which basically contains all the 1000 images used by diffusion model to generate the image corresponding to the user input.
  7. models - contains all the code of the models.
  8. scheduler - contains the code for the linear noise scheduler used for generating 1000 steps.
  9. static - it is required to for the website, it contains images that is shown on the webpage, including YAMAHA, IIT MANDI logo, and the final output image that is needed to be shown on the webpage.
  10. template - it is also required for the website, contains index.html
  11. tools- it comprises of two things, 1. subfolders for the images generated by VAE and Diffusion model, contains a subdirectory called mnist, inside which along with the previous two things, trained model ckpt files are also created. 2. it also contains code for train and sampling the model namely train_vqvae.py, train_ddpm_cond.py, sample_ddpm_cond.py
  12. utility - contains some utility code like to load/read configuration files.

Just run main.py and open localhost:5000 in the browser to start using

To Train:---

  1. Use tools.train_vqvae.py to train VAE model
  2. Once VAE is trained use tools.train_ddpm_cond.py to train the diffusion model

To Sample:----

  1. Use tools.sample_ddpm_class_cond.py for sampling the images from the model.

About

ClassyPics - Class-Conditional Image Generation: Generate images of specific objects using diffusion. Train a conditional stable diffusion model from scratch and develop an interface to input class labels and produce corresponding images.

Resources

Stars

Watchers

Forks