mist_v2.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "view-in-github"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mist-project/mist-v2/blob/main/mist_v2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5IHTCaq2fmcJ"
      },
      "source": [
        "#**Part 1: Mount the victin model**\n",
        "\n",
        "Check computing resources by running the following command. For a free user, you should see outputs similar to the following, which means you have access to one T4 GPU.\n",
        "\n",
        "\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1JaRIA6JrbkXOsE5rYpejCd3RqAdauYSA\" alt=\"image\" width=\"50%\" height=\"auto\">\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0s8xzdVPgxF1",
        "outputId": "69f6f41b-58f5-4255-aa99-29c99a42cc99"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Thu Jan 18 04:24:35 2024       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   52C    P8              12W /  70W |      0MiB / 15360MiB |      0%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "|  No running processes found                                                           |\n",
            "+---------------------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "!nvidia-smi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oRH_v2JDlUth"
      },
      "source": [
        "Create a shortcut for the victim Stable Diffusion model. Right click on the *Diffuser* folder in our [Google Drive](https://drive.google.com/drive/folders/1vg8oK2BUOla5adaJcFYx5QMq0-MoP8kk) and choose *Organize* -> *Add shortcut*.\n",
        "\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1oZ26ocm1KxK5ZmQ6GPQEHDQ2PCmtxGuH\" alt=\"image\" width=\"50%\" height=\"auto\">\n",
        "\n",
        "\n",
        "Add a shortcut of the model to *MyDrive* directory.\n",
        "\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=1FoBq2zo785dSFIfG0tSrRUoHN2kUAEd8\" alt=\"image\" width=\"50%\" height=\"auto\">"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RQ4GMIFqsgE0"
      },
      "source": [
        "Then, mount the the model to Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jBq6ZyvLfhpo",
        "outputId": "ef106303-8272-4aa8-ad6b-0901072dd657"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Mounted at /content/drive/\n"
          ]
        }
      ],
      "source": [
        "# Mount data from drive\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive/')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M9FoR_jctnu6"
      },
      "source": [
        "#**Part 2: Download Mist-v2**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jCvPqkopiuD_",
        "outputId": "e879d230-6f34-4ce6-be8c-e4aac4db466b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Cloning into 'mist-v2'...\n",
            "remote: Enumerating objects: 809, done.\u001b[K\n",
            "remote: Counting objects: 100% (84/84), done.\u001b[K\n",
            "remote: Compressing objects: 100% (70/70), done.\u001b[K\n",
            "remote: Total 809 (delta 37), reused 29 (delta 13), pack-reused 725\u001b[K\n",
            "Receiving objects: 100% (809/809), 31.30 MiB | 16.15 MiB/s, done.\n",
            "Resolving deltas: 100% (356/356), done.\n",
            "/content/mist-v2\n"
          ]
        }
      ],
      "source": [
        "!git clone https://github.com/mist-project/mist-v2.git\n",
        "%cd mist-v2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aJmDKO0ei2bq",
        "outputId": "70459f57-e986-4992-9165-23621e33d74e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu121\n",
            "Requirement already satisfied: diffusers==0.15.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 1)) (0.15.0)\n",
            "Requirement already satisfied: transformers==4.26.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 2)) (4.26.0)\n",
            "Requirement already satisfied: datasets==2.10.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 3)) (2.10.1)\n",
            "Requirement already satisfied: accelerate==0.22.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 4)) (0.22.0)\n",
            "Requirement already satisfied: gradio in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 5)) (4.14.0)\n",
            "Requirement already satisfied: ftfy in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 6)) (6.1.3)\n",
            "Requirement already satisfied: tensorboard in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 7)) (2.15.1)\n",
            "Requirement already satisfied: Jinja2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 8)) (3.1.3)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 9)) (4.66.1)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 10)) (1.11.4)\n",
            "Requirement already satisfied: fire in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 11)) (0.5.0)\n",
            "Requirement already satisfied: wandb in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 12)) (0.16.2)\n",
            "Requirement already satisfied: safetensors in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 13)) (0.4.1)\n",
            "Requirement already satisfied: opencv-python in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 14)) (4.8.0.76)\n",
            "Requirement already satisfied: mediapipe in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 15)) (0.10.9)\n",
            "Requirement already satisfied: pynvml in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 16)) (11.5.0)\n",
            "Requirement already satisfied: cache in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 17)) (1.0.3)\n",
            "Requirement already satisfied: xformers in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 18)) (0.0.23.post1)\n",
            "Requirement already satisfied: colorama in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 19)) (0.4.6)\n",
            "Requirement already satisfied: torchmetrics in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 20)) (1.3.0.post0)\n",
            "Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from diffusers==0.15.0->-r requirements.txt (line 1)) (9.4.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from diffusers==0.15.0->-r requirements.txt (line 1)) (3.13.1)\n",
            "Requirement already satisfied: huggingface-hub>=0.13.2 in /usr/local/lib/python3.10/dist-packages (from diffusers==0.15.0->-r requirements.txt (line 1)) (0.20.2)\n",
            "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.10/dist-packages (from diffusers==0.15.0->-r requirements.txt (line 1)) (7.0.1)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from diffusers==0.15.0->-r requirements.txt (line 1)) (1.23.5)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from diffusers==0.15.0->-r requirements.txt (line 1)) (2023.6.3)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from diffusers==0.15.0->-r requirements.txt (line 1)) (2.31.0)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.26.0->-r requirements.txt (line 2)) (23.2)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.26.0->-r requirements.txt (line 2)) (6.0.1)\n",
            "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.26.0->-r requirements.txt (line 2)) (0.13.3)\n",
            "Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets==2.10.1->-r requirements.txt (line 3)) (10.0.1)\n",
            "Requirement already satisfied: dill<0.3.7,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from datasets==2.10.1->-r requirements.txt (line 3)) (0.3.6)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets==2.10.1->-r requirements.txt (line 3)) (1.5.3)\n",
            "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets==2.10.1->-r requirements.txt (line 3)) (3.4.1)\n",
            "Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from datasets==2.10.1->-r requirements.txt (line 3)) (0.70.14)\n",
            "Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.10/dist-packages (from datasets==2.10.1->-r requirements.txt (line 3)) (2023.6.0)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets==2.10.1->-r requirements.txt (line 3)) (3.9.1)\n",
            "Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.10/dist-packages (from datasets==2.10.1->-r requirements.txt (line 3)) (0.18.0)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate==0.22.0->-r requirements.txt (line 4)) (5.9.5)\n",
            "Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.22.0->-r requirements.txt (line 4)) (2.1.2+cu121)\n",
            "Requirement already satisfied: aiofiles<24.0,>=22.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (23.2.1)\n",
            "Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (4.2.2)\n",
            "Requirement already satisfied: fastapi in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (0.109.0)\n",
            "Requirement already satisfied: ffmpy in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (0.3.1)\n",
            "Requirement already satisfied: gradio-client==0.8.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (0.8.0)\n",
            "Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (0.26.0)\n",
            "Requirement already satisfied: importlib-resources<7.0,>=1.3 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (6.1.1)\n",
            "Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (2.1.3)\n",
            "Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (3.7.1)\n",
            "Requirement already satisfied: orjson~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (3.9.10)\n",
            "Requirement already satisfied: pydantic>=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (2.5.3)\n",
            "Requirement already satisfied: pydub in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (0.25.1)\n",
            "Requirement already satisfied: python-multipart in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (0.0.6)\n",
            "Requirement already satisfied: semantic-version~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (2.10.0)\n",
            "Requirement already satisfied: tomlkit==0.12.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (0.12.0)\n",
            "Requirement already satisfied: typer[all]<1.0,>=0.9 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (0.9.0)\n",
            "Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (4.9.0)\n",
            "Requirement already satisfied: uvicorn>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (0.26.0)\n",
            "Requirement already satisfied: websockets<12.0,>=10.0 in /usr/local/lib/python3.10/dist-packages (from gradio-client==0.8.0->gradio->-r requirements.txt (line 5)) (11.0.3)\n",
            "Requirement already satisfied: wcwidth<0.3.0,>=0.2.12 in /usr/local/lib/python3.10/dist-packages (from ftfy->-r requirements.txt (line 6)) (0.2.13)\n",
            "Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 7)) (1.4.0)\n",
            "Requirement already satisfied: grpcio>=1.48.2 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 7)) (1.60.0)\n",
            "Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 7)) (2.17.3)\n",
            "Requirement already satisfied: google-auth-oauthlib<2,>=0.5 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 7)) (1.2.0)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 7)) (3.5.2)\n",
            "Requirement already satisfied: protobuf<4.24,>=3.19.6 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 7)) (3.20.3)\n",
            "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 7)) (67.7.2)\n",
            "Requirement already satisfied: six>1.9 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 7)) (1.16.0)\n",
            "Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 7)) (0.7.2)\n",
            "Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 7)) (3.0.1)\n",
            "Requirement already satisfied: termcolor in /usr/local/lib/python3.10/dist-packages (from fire->-r requirements.txt (line 11)) (2.4.0)\n",
            "Requirement already satisfied: Click!=8.0.0,>=7.1 in /usr/local/lib/python3.10/dist-packages (from wandb->-r requirements.txt (line 12)) (8.1.7)\n",
            "Requirement already satisfied: GitPython!=3.1.29,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb->-r requirements.txt (line 12)) (3.1.41)\n",
            "Requirement already satisfied: sentry-sdk>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb->-r requirements.txt (line 12)) (1.39.2)\n",
            "Requirement already satisfied: docker-pycreds>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from wandb->-r requirements.txt (line 12)) (0.4.0)\n",
            "Requirement already satisfied: setproctitle in /usr/local/lib/python3.10/dist-packages (from wandb->-r requirements.txt (line 12)) (1.3.3)\n",
            "Requirement already satisfied: appdirs>=1.4.3 in /usr/local/lib/python3.10/dist-packages (from wandb->-r requirements.txt (line 12)) (1.4.4)\n",
            "Requirement already satisfied: attrs>=19.1.0 in /usr/local/lib/python3.10/dist-packages (from mediapipe->-r requirements.txt (line 15)) (23.2.0)\n",
            "Requirement already satisfied: flatbuffers>=2.0 in /usr/local/lib/python3.10/dist-packages (from mediapipe->-r requirements.txt (line 15)) (23.5.26)\n",
            "Requirement already satisfied: opencv-contrib-python in /usr/local/lib/python3.10/dist-packages (from mediapipe->-r requirements.txt (line 15)) (4.8.0.76)\n",
            "Requirement already satisfied: sounddevice>=0.4.4 in /usr/local/lib/python3.10/dist-packages (from mediapipe->-r requirements.txt (line 15)) (0.4.6)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.22.0->-r requirements.txt (line 4)) (1.12)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.22.0->-r requirements.txt (line 4)) (3.2.1)\n",
            "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.22.0->-r requirements.txt (line 4)) (2.1.0)\n",
            "Requirement already satisfied: lightning-utilities>=0.8.0 in /usr/local/lib/python3.10/dist-packages (from torchmetrics->-r requirements.txt (line 20)) (0.10.0)\n",
            "Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 5)) (0.4)\n",
            "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 5)) (4.19.2)\n",
            "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 5)) (0.12.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.10.1->-r requirements.txt (line 3)) (6.0.4)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.10.1->-r requirements.txt (line 3)) (1.9.4)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.10.1->-r requirements.txt (line 3)) (1.4.1)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.10.1->-r requirements.txt (line 3)) (1.3.1)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets==2.10.1->-r requirements.txt (line 3)) (4.0.3)\n",
            "Requirement already satisfied: gitdb<5,>=4.0.1 in /usr/local/lib/python3.10/dist-packages (from GitPython!=3.1.29,>=1.0.0->wandb->-r requirements.txt (line 12)) (4.0.11)\n",
            "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 7)) (5.3.2)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 7)) (0.3.0)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 7)) (4.9)\n",
            "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<2,>=0.5->tensorboard->-r requirements.txt (line 7)) (1.3.1)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio->-r requirements.txt (line 5)) (1.2.0)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio->-r requirements.txt (line 5)) (0.12.1)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio->-r requirements.txt (line 5)) (4.47.2)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio->-r requirements.txt (line 5)) (1.4.5)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio->-r requirements.txt (line 5)) (3.1.1)\n",
            "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio->-r requirements.txt (line 5)) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets==2.10.1->-r requirements.txt (line 3)) (2023.3.post1)\n",
            "Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic>=2.0->gradio->-r requirements.txt (line 5)) (0.6.0)\n",
            "Requirement already satisfied: pydantic-core==2.14.6 in /usr/local/lib/python3.10/dist-packages (from pydantic>=2.0->gradio->-r requirements.txt (line 5)) (2.14.6)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers==0.15.0->-r requirements.txt (line 1)) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers==0.15.0->-r requirements.txt (line 1)) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers==0.15.0->-r requirements.txt (line 1)) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers==0.15.0->-r requirements.txt (line 1)) (2023.11.17)\n",
            "Requirement already satisfied: CFFI>=1.0 in /usr/local/lib/python3.10/dist-packages (from sounddevice>=0.4.4->mediapipe->-r requirements.txt (line 15)) (1.16.0)\n",
            "Requirement already satisfied: shellingham<2.0.0,>=1.3.0 in /usr/local/lib/python3.10/dist-packages (from typer[all]<1.0,>=0.9->gradio->-r requirements.txt (line 5)) (1.5.4)\n",
            "Requirement already satisfied: rich<14.0.0,>=10.11.0 in /usr/local/lib/python3.10/dist-packages (from typer[all]<1.0,>=0.9->gradio->-r requirements.txt (line 5)) (13.7.0)\n",
            "Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio->-r requirements.txt (line 5)) (0.14.0)\n",
            "Requirement already satisfied: starlette<0.36.0,>=0.35.0 in /usr/local/lib/python3.10/dist-packages (from fastapi->gradio->-r requirements.txt (line 5)) (0.35.1)\n",
            "Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio->-r requirements.txt (line 5)) (3.7.1)\n",
            "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx->gradio->-r requirements.txt (line 5)) (1.0.2)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio->-r requirements.txt (line 5)) (1.3.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata->diffusers==0.15.0->-r requirements.txt (line 1)) (3.17.0)\n",
            "Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from CFFI>=1.0->sounddevice>=0.4.4->mediapipe->-r requirements.txt (line 15)) (2.21)\n",
            "Requirement already satisfied: smmap<6,>=3.0.1 in /usr/local/lib/python3.10/dist-packages (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->-r requirements.txt (line 12)) (5.0.1)\n",
            "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 5)) (2023.12.1)\n",
            "Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 5)) (0.32.1)\n",
            "Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio->-r requirements.txt (line 5)) (0.16.2)\n",
            "Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 7)) (0.5.1)\n",
            "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard->-r requirements.txt (line 7)) (3.2.2)\n",
            "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich<14.0.0,>=10.11.0->typer[all]<1.0,>=0.9->gradio->-r requirements.txt (line 5)) (3.0.0)\n",
            "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich<14.0.0,>=10.11.0->typer[all]<1.0,>=0.9->gradio->-r requirements.txt (line 5)) (2.16.1)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx->gradio->-r requirements.txt (line 5)) (1.2.0)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate==0.22.0->-r requirements.txt (line 4)) (1.3.0)\n",
            "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich<14.0.0,>=10.11.0->typer[all]<1.0,>=0.9->gradio->-r requirements.txt (line 5)) (0.1.2)\n"
          ]
        }
      ],
      "source": [
        "!sed -i 's/cu118/cu121/g' requirements.txt\n",
        "!sed -i '/torch==.*/d' requirements.txt\n",
        "!sed -i '/torchvision==.*/d' requirements.txt\n",
        "!pip install -r requirements.txt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "4wIVSOa7zPij"
      },
      "outputs": [],
      "source": [
        "!sed -i 's/share=False/share=True/g' mist-webui.py"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "CyuBaSzEzSyT"
      },
      "outputs": [],
      "source": [
        "!mkdir /content/mist-v2/data/img\n",
        "!mkdir /content/mist-v2/data/contrast"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "dYTPvmRyzTDr"
      },
      "outputs": [],
      "source": [
        "!cp -r /content/drive/MyDrive/Diffusers/stable-diffusion-1-5 /content/mist-v2/stable-diffusion"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FyhDWw7xzfvk"
      },
      "source": [
        "#**Part 3: Boot Mist-v2**\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zdLJXIn5zTX7",
        "outputId": "088d317b-0dfa-4b90-ef05-555fa6ec19cd"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2024-01-18 04:35:48.110534: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
            "2024-01-18 04:35:48.110636: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
            "2024-01-18 04:35:48.217601: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
            "2024-01-18 04:35:48.435003: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
            "To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
            "2024-01-18 04:35:50.585614: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
            "Running on local URL:  http://127.0.0.1:7860\u001b[0m\n",
            "\u001b[0mRunning on public URL: https://c23b80fa4a3b6f09dc.gradio.live\u001b[0m\n",
            "\u001b[0m\n",
            "This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "!python mist-webui.py"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Cw--Hdv0NT0"
      },
      "source": [
        "Then, click the **public URL** to access the GUI of Mist-v2.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TDziehiq0ZAM"
      },
      "source": [
        "#**Appendix: How to upload your images and Mist them**\n",
        "\n",
        "You should upload your image to `/content/mist-v2/data/img/`. Before you run Mist-v2, make sure parameters in our GUI are set as follows:\n",
        "\n",
        "<img src=\"https://drive.google.com/uc?export=view&id=108g6dhtjxRVNMK1IqfRbZgFX4BGh0UEK\" alt=\"image\" width=\"50%\" height=\"auto\">"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "authorship_tag": "ABX9TyNvBXnHdKtbtYsC902B745N",
      "gpuType": "T4",
      "include_colab_link": true,
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}