image_to_image.ipynb

{
 "cells": [
  {
   "cell_type": "raw",
   "id": "bab46b17-057c-4981-88d7-ee5373b20bca",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "SPDX-License-Identifier: Apache-2.0\n",
    "Copyright (c) 2023, Rahul Unnikrishnan Nair <rahul.unnikrishnan.nair@intel.com>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a78a6c7-8d5d-4ac6-bb9b-93524a2a311d",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**Unleash Your Creativity: Image-to-Image Generation with Stable Diffusion**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d196333-054f-43e7-8c07-e5a65612b24d",
   "metadata": {},
   "source": [
    "Hello and welcome! Whether you're an artist, engineer, or simply someone curious about the blend of art and technology, this is the place for you.\n",
    "\n",
    "Imagine taking a picture and a few words of inspiration, and watching as they transform into a new visual representation. With Stable Diffusion, this imaginative exercise becomes a tangible experience. Want to see a landscape shift from day to night? Or perhaps turn a doodle into a detailed artwork? Dive in to explore these possibilities and more.\n",
    "\n",
    "Behind the scenes, we're harnessing the power of Intel® Data Center GPU Max Series GPUs.\n",
    "\n",
    "This guide is designed to be straightforward and user-friendly. No deep technical knowledge required, just a spark of creativity and a dash of curiosity.\n",
    "\n",
    "Ready to get started? Let's begin by setting up a few things and diving into the experience!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "68c4ee2d-e6ca-4c63-b65c-340015386d87",
   "metadata": {
    "jupyter": {
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Installation in progress...\n",
      "Installtion complete...\n"
     ]
    }
   ],
   "source": [
    "# Required packages, install if not installed (assume PyTorch* and Intel® Extension for PyTorch* is already present)\n",
    "!echo \"Installation in progress...\"\n",
    "# import sys\n",
    "# !{sys.executable} -m pip install  invisible-watermark > /dev/null\n",
    "# !conda install -y --quiet --prefix {sys.prefix}  -c conda-forge \\\n",
    "#     accelerate==0.23.0 \\\n",
    "#     validators==0.22.0 \\\n",
    "#     diffusers==0.18.2 \\\n",
    "#     transformers==4.32.1 \\\n",
    "#     tensorboardX \\\n",
    "#     pillow \\\n",
    "#     ipywidgets \\\n",
    "#     ipython > /dev/null && echo \"Installation successful\" || echo \"Installation failed\"\n",
    "import sys\n",
    "!{sys.executable} -m pip install invisible-watermark --user > /dev/null 2>&1 \n",
    "#!{sys.executable} -m pip install transformers huggingface-hub --user > /dev/null 2>&1\n",
    "!echo \"Installtion complete...\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0fe0f41a-baea-4cdf-ac53-be8700eea3e0",
   "metadata": {
    "jupyter": {
     "source_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "from io import BytesIO\n",
    "import os\n",
    "import time\n",
    "import warnings\n",
    "from pathlib import Path\n",
    "from typing import List, Dict, Tuple\n",
    "\n",
    "\n",
    "# Suppress warnings for a cleaner output.\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "import random\n",
    "import requests\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import intel_extension_for_pytorch as ipex  # adds xpu namespace to PyTorch, enabling you to use Intel GPUs\n",
    "import validators\n",
    "import numpy as np\n",
    "\n",
    "from PIL import Image\n",
    "from diffusers import StableDiffusionImg2ImgPipeline\n",
    "from diffusers import DPMSolverMultistepScheduler"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12196920-265c-4390-9fb3-5f031e7ace2d",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**A Peek Under the Hood**\n",
    "\n",
    "For those curious about how all of this works, here's a deeper dive into the code. Don't worry if you're not tech-savvy; you don't need to understand this to use the notebook. But for those interested, let's explore:\n",
    "\n",
    "- **Class Definition**: We've defined a class `Img2ImgModel` that does the heavy lifting. It sets up and optimizes the Stable Diffusion model for image transformations based on text prompts.\n",
    "  \n",
    "- **Optimization**: For best performance, the model is optimized using Intel-specific techniques in the `_optimize_pipeline` and `optimize_pipeline` methods using Intel Extension For PyTorch.\n",
    "  \n",
    "- **Image Generation**: The heart of this class, `generate_images`, takes in a text prompt and an image URL. It then generates new images based on the text's instructions, applying various parameters like strength of transformation and guidance scale.\n",
    "\n",
    "Feel free to explore the code below, and if you're inclined, you can see how we've implemented and optimized it for Intel GPUs.\n",
    "\n",
    "---\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "31324959-3ade-40d7-bbf5-feb885d931f4",
   "metadata": {
    "editable": true,
    "jupyter": {
     "source_hidden": true
    },
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "class Img2ImgModel:\n",
    "    \"\"\"\n",
    "    This class creates a model for transforming images based on given prompts.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(\n",
    "        self,\n",
    "        model_id_or_path: str,\n",
    "        device: str = \"xpu\",\n",
    "        torch_dtype: torch.dtype = torch.bfloat16,\n",
    "        optimize: bool = True,\n",
    "        warmup: bool = False,\n",
    "        scheduler: bool = True,\n",
    "    ) -> None:\n",
    "        \"\"\"\n",
    "        Initialize the model with the specified parameters.\n",
    "\n",
    "        Args:\n",
    "            model_id_or_path (str): The ID or path of the pre-trained model.\n",
    "            device (str, optional): The device to run the model on. Defaults to \"xpu\".\n",
    "            torch_dtype (torch.dtype, optional): The data type to use for the model. Defaults to torch.float16.\n",
    "            optimize (bool, optional): Whether to optimize the model. Defaults to True.\n",
    "        \"\"\"\n",
    "        self.device = device\n",
    "        self.data_type = torch_dtype\n",
    "        self.scheduler = scheduler\n",
    "        self.generator = torch.Generator()  # .manual_seed(99)\n",
    "        self.pipeline = self._load_pipeline(model_id_or_path, torch_dtype)\n",
    "        if optimize:\n",
    "            start_time = time.time()\n",
    "            #print(\"Optimizing the model...\")\n",
    "            self.optimize_pipeline()\n",
    "            #print(\n",
    "            #    \"Optimization completed in {:.2f} seconds.\".format(\n",
    "            #        time.time() - start_time\n",
    "            #    )\n",
    "            #)\n",
    "        if warmup:\n",
    "            self.warmup_model()\n",
    "\n",
    "    def _load_pipeline(\n",
    "        self, model_id_or_path: str, torch_dtype: torch.dtype\n",
    "    ) -> StableDiffusionImg2ImgPipeline:\n",
    "        \"\"\"\n",
    "        Load the pipeline for the model.\n",
    "\n",
    "        Args:\n",
    "            model_id_or_path (str): The ID or path of the pre-trained model.\n",
    "            torch_dtype (torch.dtype): The data type to use for the model.\n",
    "\n",
    "        Returns:\n",
    "            StableDiffusionImg2ImgPipeline: The loaded pipeline.\n",
    "        \"\"\"\n",
    "        print(\"Loading the model...\")\n",
    "        model_path = Path(f\"/home/common/data/Big_Data/GenAI/{model_id_or_path}\")\n",
    "        \n",
    "        if model_path.exists():\n",
    "            #print(f\"Loading the model from {model_path}...\")\n",
    "            load_path = model_path\n",
    "        else:\n",
    "            print(\"Using the default path for models...\")\n",
    "            load_path = model_id_or_path\n",
    "            \n",
    "        pipeline = StableDiffusionImg2ImgPipeline.from_pretrained(\n",
    "            load_path,\n",
    "            torch_dtype=torch_dtype,\n",
    "            use_safetensors=True,\n",
    "            variant=\"fp16\",\n",
    "        )\n",
    "        if self.scheduler:\n",
    "            pipeline.scheduler = DPMSolverMultistepScheduler.from_config(\n",
    "                pipeline.scheduler.config\n",
    "            )\n",
    "        if not model_path.exists():\n",
    "            try:\n",
    "                print(f\"Attempting to save the model to {model_path}...\")\n",
    "                pipeline.save_pretrained(f\"{model_path}\")\n",
    "                print(\"Model saved.\")\n",
    "            except Exception as e:\n",
    "                print(f\"An error occurred while saving the model: {e}. Proceeding without saving.\")\n",
    "        pipeline = pipeline.to(self.device)\n",
    "        #print(\"Model loaded.\")\n",
    "        return pipeline\n",
    "\n",
    "    \n",
    "    def _optimize_pipeline(self, pipeline: StableDiffusionImg2ImgPipeline) -> StableDiffusionImg2ImgPipeline:\n",
    "        \"\"\"\n",
    "        Optimize the pipeline of the model.\n",
    "\n",
    "        Args:\n",
    "            pipeline (StableDiffusionImg2ImgPipeline): The pipeline to optimize.\n",
    "\n",
    "        Returns:\n",
    "            StableDiffusionImg2ImgPipeline: The optimized pipeline.\n",
    "        \"\"\"\n",
    "\n",
    "        for attr in dir(pipeline):\n",
    "            try:\n",
    "                if isinstance(getattr(pipeline, attr), nn.Module):\n",
    "                    setattr(\n",
    "                        pipeline,\n",
    "                        attr,\n",
    "                        ipex.optimize(\n",
    "                            getattr(pipeline, attr).eval(),\n",
    "                            dtype=pipeline.text_encoder.dtype,\n",
    "                            inplace=True,\n",
    "                        ),\n",
    "                    )\n",
    "            except AttributeError:\n",
    "                pass\n",
    "        return pipeline\n",
    "\n",
    "    def optimize_pipeline(self) -> None:\n",
    "        \"\"\"\n",
    "        Optimize the pipeline of the model.\n",
    "        \"\"\"\n",
    "        self.pipeline = self._optimize_pipeline(self.pipeline)\n",
    "\n",
    "    def get_image_from_url(self, url: str, path: str) -> Image.Image:\n",
    "        \"\"\"\n",
    "        Get an image from a URL or from a local path if it exists.\n",
    "\n",
    "        Args:\n",
    "            url (str): The URL of the image.\n",
    "            path (str): The local path of the image.\n",
    "\n",
    "        Returns:\n",
    "            Image.Image: The loaded image.\n",
    "        \"\"\"\n",
    "        response = requests.get(url)\n",
    "        if response.status_code != 200:\n",
    "            raise Exception(\n",
    "                f\"Failed to download image. Status code: {response.status_code}\"\n",
    "            )\n",
    "        if not response.headers[\"content-type\"].startswith(\"image\"):\n",
    "            raise Exception(\n",
    "                f\"URL does not point to an image. Content type: {response.headers['content-type']}\"\n",
    "            )\n",
    "        img = Image.open(BytesIO(response.content)).convert(\"RGB\")\n",
    "        img.save(path)\n",
    "        img = img.resize((768, 512))\n",
    "        return img\n",
    "\n",
    "    def warmup_model(self):\n",
    "        \"\"\"\n",
    "        Warms up the model by generating a sample image.\n",
    "        \"\"\"\n",
    "        print(\"Setting up model...\")\n",
    "        start_time = time.time()\n",
    "        image_url = \"https://user-images.githubusercontent.com/786476/256401499-f010e3f8-6f8d-4e9f-9d1f-178d3571e7b9.png\"\n",
    "        try:\n",
    "            self.generate_images(\n",
    "                image_url=image_url,\n",
    "                prompt=\"A beautiful day\",\n",
    "                num_images=1,\n",
    "                save_path=\".tmp\",\n",
    "            )\n",
    "        except Exception:\n",
    "            print(\"model warmup delayed...\")\n",
    "        #print(\n",
    "        #    \"Model is set up and ready! Warm-up completed in {:.2f} seconds.\".format(\n",
    "        #        time.time() - start_time\n",
    "        #    )\n",
    "        #)\n",
    "\n",
    "    def get_inputs(self, prompt, batch_size=1):\n",
    "        self.generator = [torch.Generator() for i in range(batch_size)]\n",
    "        prompts = batch_size * [prompt]\n",
    "        return {\"prompt\": prompts, \"generator\": self.generator}\n",
    "\n",
    "    def generate_images(\n",
    "        self,\n",
    "        prompt: str,\n",
    "        image_url: str,\n",
    "        num_images: int = 5,\n",
    "        num_inference_steps: int = 30,\n",
    "        strength: float = 0.75,\n",
    "        guidance_scale: float = 7.5,\n",
    "        save_path: str = \"image_to_image\",\n",
    "        batch_size: int = 1,\n",
    "    ):\n",
    "        \"\"\"\n",
    "        Generate images based on the provided prompt and variations.\n",
    "\n",
    "        Args:\n",
    "            prompt (str): The base prompt for the generation.\n",
    "            image_url (str): The URL of the seed image.\n",
    "            variations (List[str]): The list of variations to apply to the prompt.\n",
    "            num_images (int, optional): The number of images to generate. Defaults to 5.\n",
    "            num_inference_steps (int, optional): Number of noise removal steps.\n",
    "            strength (float, optional): The strength of the transformation. Defaults to 0.75.\n",
    "            guidance_scale (float, optional): The scale of the guidance. Defaults to 7.5.\n",
    "            save_path (str, optional): The path to save the generated images. Defaults to \"image_to_image\".\n",
    "\n",
    "        \"\"\"\n",
    "        input_image_path = \"input.png\"\n",
    "        init_image = self.get_image_from_url(image_url, input_image_path)\n",
    "        init_images = [init_image for _ in range(batch_size)]\n",
    "        for i in range(0, num_images, batch_size):\n",
    "            with torch.xpu.amp.autocast(\n",
    "                enabled=True if self.data_type != torch.float32 else False,\n",
    "                dtype=self.data_type,\n",
    "            ):\n",
    "                if batch_size > 1:\n",
    "                    inputs = self.get_inputs(batch_size=batch_size, prompt=prompt)\n",
    "                    images = self.pipeline(\n",
    "                        **inputs,\n",
    "                        image=init_images,\n",
    "                        strength=strength,\n",
    "                        guidance_scale=guidance_scale,\n",
    "                        num_inference_steps=num_inference_steps,\n",
    "                    ).images\n",
    "                else:\n",
    "                    images = self.pipeline(\n",
    "                        prompt=prompt,\n",
    "                        image=init_images,\n",
    "                        strength=strength,\n",
    "                        guidance_scale=guidance_scale,\n",
    "                        num_inference_steps=num_inference_steps,\n",
    "                    ).images\n",
    "\n",
    "                for j in range(len(images)):\n",
    "                    output_image_path = os.path.join(\n",
    "                        save_path,\n",
    "                        f\"{'_'.join(prompt.split()[:3])}_{i+j}__{int(time.time() * 1e6)}.png\",\n",
    "                    )\n",
    "                    images[j].save(output_image_path)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "478a838e-3032-4f4b-ac23-7ef142d20d77",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**Setting Up the User Interface**\n",
    "\n",
    "In the next section, we'll craft an interactive user interface right here in the notebook. This will allow you to easily select a model, provide an image URL, type in a text prompt, and define other parameters without diving into the code itself.\n",
    "\n",
    "- **Model Selection**: Choose from available pre-trained models.\n",
    "- **Prompt Input**: Type in a text prompt to guide the image transformation.\n",
    "- **Number of Images**: Decide how many images you'd like to generate.\n",
    "- **Image URL**: Provide a link to an online image or use the default provided.\n",
    "- **Enhancement**: Opt for auto-enhancements to the prompt for added creativity.\n",
    "\n",
    "Once you've provided your preferences, a button click will initiate the magic!\n",
    "\n",
    "Let's set this up:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e012d657-7fe4-426e-bdeb-033701a38a52",
   "metadata": {
    "jupyter": {
     "source_hidden": true
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import random\n",
    "import time\n",
    "\n",
    "import ipywidgets as widgets\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as mp_img\n",
    "import validators\n",
    "\n",
    "from IPython.display import clear_output\n",
    "from IPython.display import display\n",
    "from IPython.display import HTML\n",
    "from IPython.display import Image as IPImage\n",
    "from ipywidgets import VBox, HBox\n",
    "\n",
    "model_cache = {}\n",
    "\n",
    "def image_to_image():\n",
    "    out = widgets.Output()\n",
    "    image_to_image_dir = \"image_to_image\"\n",
    "    num_images = 2\n",
    "    model_ids = [\n",
    "        \"runwayml/stable-diffusion-v1-5\",\n",
    "        \"stabilityai/stable-diffusion-2-1\",\n",
    "    ]    \n",
    "    model_dropdown = widgets.Dropdown(\n",
    "        options=model_ids,\n",
    "        value=model_ids[0],\n",
    "        description=\"Model:\",\n",
    "    )    \n",
    "    prompt_text = widgets.Text(\n",
    "        value=\"\",\n",
    "        placeholder=\"Enter your prompt\",\n",
    "        description=\"Prompt:\",\n",
    "    )    \n",
    "    num_images_slider = widgets.IntSlider(\n",
    "        value=2,\n",
    "        min=1,\n",
    "        max=10,\n",
    "        step=1,\n",
    "        description=\"Images:\",\n",
    "    )    \n",
    "    image_url_text = widgets.Text(\n",
    "        value=\"https://user-images.githubusercontent.com/786476/256401499-f010e3f8-6f8d-4e9f-9d1f-178d3571e7b9.png\",\n",
    "        placeholder=\"Enter an image URL\",\n",
    "        description=\"Image URL:\",\n",
    "    )\n",
    "    enhance_checkbox = widgets.Checkbox(\n",
    "        value=False,\n",
    "        description=\"Auto enhance the prompt?\",\n",
    "        disabled=False,\n",
    "        indent=False\n",
    "    )\n",
    "    enhance_checkbox.layout.margin = \"0 0 0 10px\"\n",
    "    num_images_slider.layout.margin = \"0 0 0 8px\"\n",
    "    prompt_text.layout.width = \"100%\"\n",
    "    layout = widgets.Layout(margin=\"0px 50px 10px 0px\")\n",
    "    button = widgets.Button(description=\"Generate Images!\", button_style=\"primary\")\n",
    "    left_box = VBox([model_dropdown,num_images_slider], layout=layout)\n",
    "    right_box = VBox([image_url_text, enhance_checkbox], layout=layout)\n",
    "    user_input_widgets = HBox([left_box, right_box], layout=layout)\n",
    "    prompt_text.layout.width = \"57.5%\"\n",
    "    button.layout.margin = \"35px\"\n",
    "    display(user_input_widgets)\n",
    "    display( prompt_text)\n",
    "    display(button)\n",
    "    display(out)\n",
    "    \n",
    "    \n",
    "    def on_submit(button):\n",
    "        with out:\n",
    "            clear_output(wait=True)\n",
    "            print(\"Once generated, images will be saved to `./image_to_image` dir, please wait...\")\n",
    "            selected_model_index = model_ids.index(model_dropdown.value)\n",
    "            model_id = model_ids[selected_model_index]\n",
    "            model_key = (model_id, \"xpu\")\n",
    "            prompt = prompt_text.value\n",
    "            num_images = num_images_slider.value\n",
    "            image_url = image_url_text.value\n",
    "            \n",
    "            if not validators.url(image_url):\n",
    "                print(\"The input is not a valid URL. Using the default URL instead.\")\n",
    "                image_url = \"https://user-images.githubusercontent.com/786476/256401499-f010e3f8-6f8d-4e9f-9d1f-178d3571e7b9.png\"       \n",
    "            #model = Img2ImgModel(model_id, device=\"xpu\")\n",
    "            if model_key not in model_cache:\n",
    "                model_cache[model_key] = Img2ImgModel(model_id, device=\"xpu\")\n",
    "            model = model_cache[model_key]\n",
    "            enhancements = [\n",
    "            \"purple light\",\n",
    "            \"dreaming\",\n",
    "            \"cyberpunk\",\n",
    "            \"ancient\" \", rustic\",\n",
    "            \"gothic\",\n",
    "            \"historical\",\n",
    "            \"punchy\",\n",
    "            \"photo\" \"vivid colors\",\n",
    "            \"4k\",\n",
    "            \"bright\",\n",
    "            \"exquisite\",\n",
    "            \"painting\",\n",
    "            \"art\",\n",
    "            \"fantasy [,/organic]\",\n",
    "            \"detailed\",\n",
    "            \"trending in artstation fantasy\",\n",
    "            \"electric\",\n",
    "            \"night\",\n",
    "            ]\n",
    "            if not prompt:\n",
    "                prompt = \" \"\n",
    "            if enhance_checkbox.value:\n",
    "                prompt = prompt + \" \" + \" \".join(random.sample(enhancements, 5))\n",
    "                print(f\"Using enhanced prompt: {prompt}\")    \n",
    "            try:\n",
    "                start_time = time.time()\n",
    "                os.makedirs(image_to_image_dir, exist_ok=True)\n",
    "                model.generate_images(\n",
    "                    prompt=prompt,\n",
    "                    image_url=image_url,\n",
    "                    num_images=num_images,\n",
    "                )\n",
    "                clear_output(wait=True)\n",
    "                display_generated_images()\n",
    "            except KeyboardInterrupt:\n",
    "                print(\"\\nUser interrupted image generation...\")\n",
    "            except Exception as e:\n",
    "                print(f\"An error occurred: {e}\")\n",
    "            finally:\n",
    "                status = f\"Complete generating {num_images} images in {time.time() - start_time:.2f} seconds.\"\n",
    "                #print(status)\n",
    "    button.on_click(on_submit)\n",
    "\n",
    "def display_generated_images(image_to_image_dir=\"image_to_image\"):\n",
    "    image_files = [f for f in os.listdir(image_to_image_dir) if f.endswith((\".png\", \".jpg\"))]    \n",
    "    num_images = len(image_files)\n",
    "    num_columns = int(np.ceil(np.sqrt(num_images)))\n",
    "    num_rows = int(np.ceil(num_images / num_columns))\n",
    "    fig, axs = plt.subplots(num_rows, num_columns, figsize=(10 * num_columns / num_columns, 10 * num_rows / num_rows))\n",
    "    if num_images == 1:\n",
    "        axs = np.array([[axs]])\n",
    "    elif num_columns == 1 or num_rows == 1:\n",
    "        axs = np.array([axs])\n",
    "    for ax, image_file in zip(axs.ravel(), image_files):\n",
    "        img = mp_img.imread(os.path.join(image_to_image_dir, image_file))\n",
    "        ax.imshow(img)\n",
    "        ax.axis(\"off\")  # Hide axes\n",
    "    for ax in axs.ravel()[num_images:]:\n",
    "        ax.axis(\"off\")\n",
    "    plt.tight_layout()\n",
    "    print(f\"\\nGenerated images...:\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0da73551-7707-46ac-884f-0b0f6138280b",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**Let's Dive In! and Witness the Magic**\n",
    "\n",
    "Ready to generate some amazing images? Just interact with the user interface below. Once you've set your preferences, click the \"Generate Images!\" button to witness the power of Stable Diffusion in action.\n",
    "\n",
    "Once done, you'll find the images generated based on your prompt. Presented in a grid format, they showcase the diverse interpretations and transformations the model inferred from your input.\n",
    "\n",
    "\n",
    "Go ahead, unleash your creativity!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "39941920-7519-4dd3-8e59-37c8f1389dd1",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "99b7064043994ec8acfe66e26f093942",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(VBox(children=(Dropdown(description='Model:', options=('runwayml/stable-diffusion-v1-5', 'stabi…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f918ca1e92894356bf3e5174c1402022",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Text(value='', description='Prompt:', layout=Layout(width='57.5%'), placeholder='Enter your prompt')"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "48e0834b880f4ae18d911649c683e12b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Button(button_style='primary', description='Generate Images!', layout=Layout(margin='35px'), style=ButtonStyle…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "212bd39aabd44260a96a10c2dbf928d5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Run all cells before running this section and wait a few seconds for UI to load\n",
    "image_to_image()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1915e149-0ad6-475d-9b16-b34eb3d2806a",
   "metadata": {},
   "source": [
    "#### Reference and Guidelines for Models Used in This Notebook\n",
    "\n",
    "\n",
    "##### runwayml/stable-diffusion-v1-5\n",
    "- **Model card:** [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5)\n",
    "- **License:** CreativeML OpenRAIL M license\n",
    "- **Reference:**\n",
    "    ```bibtex\n",
    "    @InProceedings{Rombach_2022_CVPR,\n",
    "        author    = {Rombach, Robin and Blattmann, Andreas and Lorenz, Dominik and Esser, Patrick and Ommer, Bj\\\"orn},\n",
    "        title     = {High-Resolution Image Synthesis With Latent Diffusion Models},\n",
    "        booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n",
    "        month     = {June},\n",
    "        year      = {2022},\n",
    "        pages     = {10684-10695}\n",
    "    }\n",
    "    ```\n",
    "\n",
    "##### stabilityai/stable-diffusion-2-1\n",
    "- **Model card:** [stabilityai/stable-diffusion-2-1](https://huggingface.co/stabilityai/stable-diffusion-2-1)\n",
    "- **License:** CreativeML Open RAIL++-M License\n",
    "- **Reference:**\n",
    "    ```bibtex\n",
    "    @InProceedings{Rombach_2022_CVPR,\n",
    "        author    = {Rombach, Robin and Blattmann, Andreas and Lorenz, Dominik and Esser, Patrick and Ommer, Bj\\\"orn},\n",
    "        title     = {High-Resolution Image Synthesis With Latent Diffusion Models},\n",
    "        booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n",
    "        month     = {June},\n",
    "        year      = {2022},\n",
    "        pages     = {10684-10695}\n",
    "    }\n",
    "    ```\n",
    "\n",
    "##### Disclaimer for Using Stable Diffusion Models\n",
    "\n",
    "The stable diffusion models provided here are powerful tools for high-resolution image synthesis, including text-to-image and image-to-image transformations. While they are designed to produce high-quality results, users should be aware of potential limitations:\n",
    "\n",
    "- **Quality Variation:** The quality of generated images may vary based on the complexity of the input text or image, and the alignment with the model's training data.\n",
    "- **Licensing and Usage Constraints:** Please carefully review the licensing information associated with each model to ensure compliance with all terms and conditions.\n",
    "- **Ethical Considerations:** Consider the ethical implications of the generated content, especially in contexts that may involve sensitive or controversial subjects.\n",
    "\n",
    "For detailed information on each model's capabilities, limitations, and best practices, please refer to the respective model cards and associated publications linked above.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python-gpu",
   "language": "python",
   "name": "python-gpu"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}