Image Diffusion with CLIP

This repository contains Python code for using a CLIP model to perform image diffusion

Example 1	Example 2

This repository contains Python code for using a CLIP model to perform image diffusion. Specifically, it uses the Stable Diffusion algorithm to transform an input image according to a given prompt, while also using a negative prompt to ensure that the output is not too dissimilar from the input.

The code requires several dependencies, including the Pillow and PyTorch libraries, as well as the CLIP Interrogator and Diffusers packages. To run the code, you will also need to have a pre-trained Stable Diffusion model, which can be downloaded from the StabilityAI Model Hub.

Example 3	Example 4

The main entry point for the code is the interrogate_images function, which takes two image paths as input and returns two lists of prompts generated by the CLIP model. These prompts can then be used to perform image diffusion using the StableDiffusionImg2ImgPipeline class from the Diffusers package. The resulting image is saved as a PNG file.

This project is an implementation of a stable diffusion model, which is a machine learning algorithm that generates new images based on textual prompts. It is an analog to the popular MidJourney Remix, which uses a similar approach to generate images based on input text. This implementation uses the Stable Diffusion pipeline, which is designed to generate high-quality images and is particularly effective at image-to-image translation tasks. The pipeline is pretrained on a large dataset of images and prompts, and can be fine-tuned on specific tasks with additional data.

To run the script with two image paths, open the command line interface and navigate to the directory where the script is located. Then run the following command:

bash
python run.py path/to/image1.png path/to/image2.png

Replace run.py, path/to/image1.png, and path/to/image2.png with the actual names and paths of the script and images, respectively.

Note: Make sure that you have the required libraries and dependencies installed before running the script. You can refer to the requirements.txt file for a list of required libraries and their versions.

This code was developed by Alexey Khaneuski.