MUNIT.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "MUNIT.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/dvschultz/MUNIT/blob/master/MUNIT.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IdC23munnMbJ"
      },
      "source": [
        "#MUNIT\n",
        "MUNIT stands for Multimodal UNsupervised Image-to-image Translation. That’s a lot of words to say it can convert images of cats to images of dogs, or images of horses to images of zebras. Basically, it learns to convert one set of images (a “domain”) to another set.\n",
        "\n",
        "[Original repo](https://github.com/NVlabs/MUNIT) | [Forked repo](https://github.com/dvschultz/MUNIT)\n",
        "\n",
        "There is also a new version of MUNIT in the Imaginaire library. I personally didn’t like it as much but YMMV.\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "\n",
        "If you find this notebook useful, consider signing up for my [Patreon](https://www.patreon.com/bustbright) or [YouTube channel](https://www.youtube.com/channel/UCaZuPdmZ380SFUMKHVsv_AA/join). You can also send me a one-time payment on [Venmo](https://venmo.com/Derrick-Schultz)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Szmde0mW5eLt"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w1FfZ_T85y_H"
      },
      "source": [
        "First let’s check to see what GPU we’ve been assigned."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CUS9o_ko5c_3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "cde9f410-9d88-48d6-971e-109fb64b694f"
      },
      "source": [
        "!nvidia-smi -L"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "GPU 0: Tesla V100-SXM2-16GB (UUID: GPU-1799cab7-47ba-967b-96b1-38b019c5ec57)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E_IQc0FQ525y"
      },
      "source": [
        "Next let’s connect our Google Drive account to Colab. This is optional but highly recommended."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hr-hnVHv5gO4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3d5a2a77-62c2-4a47-a4f7-399ee04ed375"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q7kIYZQc5apQ"
      },
      "source": [
        "## Install repo"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GI6Erd5s7YO0"
      },
      "source": [
        "The next cell will install the MUNIT repository in Google Drive. If you have already installed it it will just move into that folder. If you don’t have Google Drive connected it will just install the necessary code in Colab."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "epYHn8rR42aR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "62cd8f24-5afe-40c8-fc70-a9d14517eea7"
      },
      "source": [
        "import os\n",
        "\n",
        "if os.path.isdir(\"/content/drive/MyDrive/colab-munit/MUNIT/\"):\n",
        "    %cd /content/drive/MyDrive/colab-munit/MUNIT/\n",
        "elif os.path.isdir(\"/content/drive/\"):\n",
        "    !mkdir /content/drive/MyDrive/colab-munit\n",
        "    %cd /content/drive/MyDrive/colab-munit\n",
        "    !git clone https://github.com/dvschultz/MUNIT\n",
        "    %cd MUNIT\n",
        "else:\n",
        "    print('Drive not mounted, installing in /content/')  \n",
        "    !git clone https://github.com/dvschultz/MUNIT\n",
        "    %cd MUNIT\n",
        "\n",
        "#install dependencies\n",
        "!pip install tensorboardX torchfile"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/content/drive/MyDrive/colab-munit/MUNIT\n",
            "Requirement already satisfied: tensorboardX in /usr/local/lib/python3.7/dist-packages (2.5)\n",
            "Requirement already satisfied: torchfile in /usr/local/lib/python3.7/dist-packages (0.1.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from tensorboardX) (1.21.5)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from tensorboardX) (1.15.0)\n",
            "Requirement already satisfied: protobuf>=3.8.0 in /usr/local/lib/python3.7/dist-packages (from tensorboardX) (3.17.3)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J4me0Dxf2uZK",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "df296978-3997-46d3-cd0a-8a470c08b247"
      },
      "source": [
        "%cd \"/content/drive/MyDrive/colab-munit/MUNIT/\"\n",
        "!git config --global user.name \"test\"\n",
        "!git config --global user.email \"test@test.com\"\n",
        "!git fetch origin\n",
        "!git pull\n",
        "!git checkout origin/master -- test.py test_batch.py"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/drive/MyDrive/colab-munit/MUNIT\n",
            "Already up to date.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "My_COMHzkoYm"
      },
      "source": [
        "##Dataset Preparation\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w_aLCLwXewMC"
      },
      "source": [
        "MUNIT requires two different sets of images. Unlike Pix2Pix or similar models, these two sets of images do not need to be tightly coupled (for example: a color image and its grayscale equivalent). The images also do not need to be the same size.\n",
        "\n",
        "I recommend a minimum of ~250 images per folder. This will allow for the model to generalize fairly well (the more images, the better). \n",
        "\n",
        "Once you have created the images you will need to upload them to your server. The folder structure should look like this:\n",
        "* Top level folder name (I usually use X2Y, where X is the first dataset and Y is the second.)\n",
        "* ├ trainA (training set from X)\n",
        "* ├ trainB (training set from Y)\n",
        "* ├ testA (testing set from X)\n",
        "* ├ testB (testing set from Y)\n",
        "\n",
        "For our purposes `trainA`/`testA` and `trainB`/`testB` can be the same, but in a \"scientific\" setting you would want these folder to contain different images.\n",
        "\n",
        "There are two ways to upload the images.\n",
        "1. Zip up the folder and upload it to Colab in the `MUNIT/datasets` folder. This is probably slow and depending how large your dataset is can use up a lot of the Disk space in Colab.\n",
        "2. You can connect your Google Drive to Colab and host your files in Google Drive (This does not require you zipping up your folder, but does require a Google Drive account). This option needs a little setting up, but it saves you disk space and time once it’s set up."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7AsDwtVNkl39"
      },
      "source": [
        "#Training\n",
        "Once you have your dataset prepped and uploaded, the last thing to do is create a YAML config file. In the code below I recommend renaming the file to match the top level folder name you used when uploading the dataset. This helps keep track of file naming and will remind you what settings you used should you need to retrain the model ever again."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lzFp9wkt74b8"
      },
      "source": [
        "* `data_root`: should point to the path of your dataset\n",
        "* `new_size`, `crop_image_height`, `crop_image_width` (optional): I recommend setting this to a larger size, I find it leads to better training (I usually use `1024`, `512`, and `512` respectively)."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%writefile ./configs/birds2floralmag_folder.yaml\n",
        "\n",
        "# Copyright (C) 2017 NVIDIA Corporation.  All rights reserved.\n",
        "# Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).\n",
        "\n",
        "# logger options\n",
        "image_save_iter: 10000        # How often do you want to save output images during training\n",
        "image_display_iter: 500       # How often do you want to display output images during training\n",
        "display_size: 16              # How many images do you want to display each time\n",
        "snapshot_save_iter: 10000     # How often do you want to save trained models\n",
        "log_iter: 10                  # How often do you want to log the training stats\n",
        "\n",
        "# optimization options\n",
        "max_iter: 1000000             # maximum number of training iterations\n",
        "batch_size: 1                 # batch size\n",
        "weight_decay: 0.0001          # weight decay\n",
        "beta1: 0.5                    # Adam parameter\n",
        "beta2: 0.999                  # Adam parameter\n",
        "init: kaiming                 # initialization [gaussian/kaiming/xavier/orthogonal]\n",
        "lr: 0.0001                    # initial learning rate\n",
        "lr_policy: step               # learning rate scheduler\n",
        "step_size: 100000             # how often to decay learning rate\n",
        "gamma: 0.5                    # how much to decay learning rate\n",
        "gan_w: 1                      # weight of adversarial loss\n",
        "recon_x_w: 10                 # weight of image reconstruction loss\n",
        "recon_s_w: 1                  # weight of style reconstruction loss\n",
        "recon_c_w: 1                  # weight of content reconstruction loss\n",
        "recon_x_cyc_w: 0              # weight of explicit style augmented cycle consistency loss\n",
        "vgg_w: 0                      # weight of domain-invariant perceptual loss\n",
        "\n",
        "# model options\n",
        "gen:\n",
        "  dim: 64                     # number of filters in the bottommost layer\n",
        "  mlp_dim: 256                # number of filters in MLP\n",
        "  style_dim: 8                # length of style code\n",
        "  activ: relu                 # activation function [relu/lrelu/prelu/selu/tanh]\n",
        "  n_downsample: 2             # number of downsampling layers in content encoder\n",
        "  n_res: 4                    # number of residual blocks in content encoder/decoder\n",
        "  pad_type: reflect           # padding type [zero/reflect]\n",
        "dis:\n",
        "  dim: 64                     # number of filters in the bottommost layer\n",
        "  norm: none                  # normalization layer [none/bn/in/ln]\n",
        "  activ: lrelu                # activation function [relu/lrelu/prelu/selu/tanh]\n",
        "  n_layer: 4                  # number of layers in D\n",
        "  gan_type: lsgan             # GAN loss [lsgan/nsgan]\n",
        "  num_scales: 3               # number of scales\n",
        "  pad_type: reflect           # padding type [zero/reflect]\n",
        "\n",
        "# data options\n",
        "input_dim_a: 3                              # number of image channels [1/3]\n",
        "input_dim_b: 3                              # number of image channels [1/3]\n",
        "num_workers: 12                              # number of data loading threads\n",
        "new_size: 1024                               # first resize the shortest image side to this size\n",
        "crop_image_height: 256                      # random crop image of this height\n",
        "crop_image_width: 256                       # random crop image of this width\n",
        "data_root: /content/drive/MyDrive/MUNIT/datasets/birds2floralmag     # dataset folder location"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4STEOsBFlkp5",
        "outputId": "48c9b982-da25-4dcf-a1e2-4ca461fdf203"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Overwriting ./configs/birds2floralmag_folder.yaml\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k30-Rcuj9BQo"
      },
      "source": [
        "Ok, now it’s time to train. We can see all of the options for training by running the next cell."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vw06b6id9G7J",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0a83d0b0-c920-4407-a5f4-8fbc56b3ee57"
      },
      "source": [
        "!python train.py --help"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "usage: train.py [-h] [--config CONFIG] [--output_path OUTPUT_PATH] [--resume]\n",
            "                [--trainer TRAINER]\n",
            "\n",
            "optional arguments:\n",
            "  -h, --help            show this help message and exit\n",
            "  --config CONFIG       Path to the config file.\n",
            "  --output_path OUTPUT_PATH\n",
            "                        outputs path\n",
            "  --resume\n",
            "  --trainer TRAINER     MUNIT|UNIT\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OVd31o4t9fU4"
      },
      "source": [
        "In my experience all we really need to worry about it the `--config` argument."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2Anm_z5Djolg"
      },
      "source": [
        "!python train.py --config ./configs/birds2floralmag_folder.yaml"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lF34U7ly8nqS"
      },
      "source": [
        "### Resume Training\n",
        "\n",
        "If Colab shuts off before you complete training, you can resume training with this command:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ywU6kk9p89bc"
      },
      "source": [
        "!python train.py --config ./configs/birds2floralmag_folder.yaml --resume"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P-MJQVQmo7RB"
      },
      "source": [
        "# Testing\n",
        "\n",
        "These steps assume you have already trained a model in Colab. If you have the model somewhere else, skip down to the `Testing (Alternate)` section."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EvlasE49pG4u"
      },
      "source": [
        "## Single Image Testing\n",
        "\n",
        "`--config`: This should be the path to the same config file you trained with.\n",
        "\n",
        "`--input`: path to a single image you want to use as your input image\n",
        "\n",
        "`--output_folder`: path to save output images\n",
        "\n",
        "`--checkpoint`: path to the checkpoint you want to use as the model\n",
        "\n",
        "`--a2b`: set to `1` if you want a2b inference, set to `0` if you want b2a."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vKm-UoVMpUXQ"
      },
      "source": [
        "#testing with a single image\n",
        "#this generates 10 random images from the model\n",
        "!python test.py \\\n",
        "--config /content/drive/MyDrive/colab-munit/MUNIT/configs/1024birds2floralmag.yaml \\\n",
        "--checkpoint /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/checkpoints/gen_00060000.pt \\\n",
        "--input /content/drive/MyDrive/colab-munit/MUNIT/datasets/birds2floralmag/testA/birdsAustraliav5Goul_0232.png \\\n",
        "--output_folder /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/inference/ \\\n",
        "--a2b 1 #switch to 0 if you want b2a"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yMiQ2weaqLDT"
      },
      "source": [
        "## Additional Single Image Testing Options\n",
        "`--num_style`: How many \"styles\" you want to output\n",
        "\n",
        "`--num_style_start`: What index to start with. If you combine this with `--num_style 1` you can generate only the style you want once determined (for example, run 500 samples, pick your favorite, then only ever test with that one.)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xpsf2HS_qV_j"
      },
      "source": [
        "!python test.py \\\n",
        "--config /content/drive/MyDrive/colab-munit/MUNIT/configs/1024birds2floralmag.yaml \\\n",
        "--checkpoint /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/checkpoints/gen_00060000.pt \\\n",
        "--input /content/drive/MyDrive/colab-munit/MUNIT/datasets/birds2floralmag/testA/birdsAustraliav5Goul_0232.png \\\n",
        "--output_folder /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/inference/ \\\n",
        "--a2b 1 \\\n",
        "--num_style 1 \\\n",
        "--num_style_start 65"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jh1VTWClqedi"
      },
      "source": [
        "##Guided Translation\n",
        "MUNIT also allows you to do what they call \"guided translation.\" This allows you to pass it an image and use that image as the \"style\" (for lack of a better term) for the output image.\n",
        "\n",
        "`--style`: path to an image to use as the guided style. (Note: this isn’t like style transfer where it can be an arbitrary image. It should be an image from your training set)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oaTWIHDhqfiR"
      },
      "source": [
        "#guided translation\n",
        "!python test.py \\\n",
        "--config /content/drive/MyDrive/colab-munit/MUNIT/configs/1024birds2floralmag.yaml \\\n",
        "--checkpoint /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/checkpoints/gen_00060000.pt \\\n",
        "--input /content/drive/MyDrive/colab-munit/MUNIT/datasets/birds2floralmag/testA/birdsAustraliav5Goul_0232.png \\\n",
        "--output_folder /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/inference/ \\\n",
        "--a2b 1 \\\n",
        "--style ./datasets/tjpd2seguy/testB/089.JPG"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HDg1otZRq2cu"
      },
      "source": [
        "##Interpolation between styles\n",
        "Lastly we can also animate between styles, similar to an interpolation in other GAN models like StyleGAN.\n",
        "\n",
        "`--interpolation`: add this to enable interpolation mode\n",
        "\n",
        "`--seeds`: specify which seeds you want to interpolate between\n",
        "\n",
        "`--frames`: how many frames for the total interpolation\n",
        "\n",
        "*One thing to note that might not be obvious:* `--num_style` must be >= the largest value set in `--seeds`. If you use `--seeds 0,22,35` you need `--num_style 36`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tMjq4gOWsQ4a"
      },
      "source": [
        "!python test.py \\\n",
        "--config /content/drive/MyDrive/colab-munit/MUNIT/configs/1024birds2floralmag.yaml \\\n",
        "--checkpoint /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/checkpoints/gen_00060000.pt \\\n",
        "--input /content/drive/MyDrive/colab-munit/MUNIT/datasets/birds2floralmag/testA/birdsAustraliav5Goul_0232.png \\\n",
        "--output_folder /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/inference-interpolation/ \\\n",
        "--a2b 1 \\\n",
        "--interpolation \\\n",
        "--seeds 0,99,9,50,25,0 \\\n",
        "--num_style 100 \\\n",
        "--frames 720"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rCquciaRwPry"
      },
      "source": [
        "You can turn these frames into a video using the following command:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h9YFvhuLwVLW"
      },
      "source": [
        "!ffmpeg -y -r 24 -i /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/inference-interpolation//output%03d.png -vcodec libx264 -pix_fmt yuv420p /content/munit-interpolation.mp4"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yvQ4Esqm2Q6M"
      },
      "source": [
        "## Batch Image Testing"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QU7HVNWt2aNj"
      },
      "source": [
        "!python test_batch.py --help"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1ZKBgK9Z4Jx8"
      },
      "source": [
        "### Batch Processing\n",
        "\n",
        "`--input`: specify a folder path as the input\n",
        "\n",
        "`--output`: specify a folder path as the output to save images"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "06AlDDQL2Y3w"
      },
      "source": [
        "!python test_batch.py \\\n",
        "--config /content/drive/MyDrive/colab-munit/MUNIT/configs/1024birds2floralmag.yaml \\\n",
        "--checkpoint /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/checkpoints/gen_00060000.pt \\\n",
        "--input_folder /content/drive/MyDrive/colab-munit/MUNIT/datasets/birds2floralmag/testA/ \\\n",
        "--output_folder /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/batch-inference/ \\\n",
        "--a2b 1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6JlWi9RL4Cnf"
      },
      "source": [
        "### Video/Interpolation Processing\n",
        "\n",
        "`--video`: use this argument to specify you want to use the `--input` folder as a series of frames from a video\n",
        "\n",
        "`--seeds`: like with the interpolation method on single images, you need to pass in a series of seeds "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7AkEw5AAFEYT"
      },
      "source": [
        "!mkdir /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/batch-inference/outputs-batch-video"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hUQdNBXi8Y_B"
      },
      "source": [
        "!python test_batch.py \\\n",
        "--config /content/drive/MyDrive/colab-munit/MUNIT/configs/1024birds2floralmag.yaml \\\n",
        "--checkpoint /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/checkpoints/gen_00060000.pt \\\n",
        "--input_folder ./1080/ \\\n",
        "--output_folder /content/drive/MyDrive/colab-munit/MUNIT/outputs/1024birds2floralmag/outputs-batch-video/ \\\n",
        "--a2b 1 \\\n",
        "--video \\\n",
        "--seeds 0,1,2,3,4,5,4,3,2,1,0"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NkxVAHwGBn0u",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "51293e6e-1db5-4e30-eb00-49f29e5da8b1"
      },
      "source": [
        "!ffmpeg -y -r 24 -i /content/drive/MyDrive/colab-munit/MUNIT/outputs-batch-video/output%09d.png -vcodec libx264 -pix_fmt yuv420p /content/munit-interpolation-batch.mp4"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "ffmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers\n",
            "  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)\n",
            "  configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared\n",
            "  libavutil      55. 78.100 / 55. 78.100\n",
            "  libavcodec     57.107.100 / 57.107.100\n",
            "  libavformat    57. 83.100 / 57. 83.100\n",
            "  libavdevice    57. 10.100 / 57. 10.100\n",
            "  libavfilter     6.107.100 /  6.107.100\n",
            "  libavresample   3.  7.  0 /  3.  7.  0\n",
            "  libswscale      4.  8.100 /  4.  8.100\n",
            "  libswresample   2.  9.100 /  2.  9.100\n",
            "  libpostproc    54.  7.100 / 54.  7.100\n",
            "Input #0, image2, from '/content/drive/MyDrive/colab-munit/MUNIT/outputs-batch-video/output%09d.png':\n",
            "  Duration: 00:00:28.80, start: 0.000000, bitrate: N/A\n",
            "    Stream #0:0: Video: png, rgb24(pc), 720x720, 25 fps, 25 tbr, 25 tbn, 25 tbc\n",
            "Stream mapping:\n",
            "  Stream #0:0 -> #0:0 (png (native) -> h264 (libx264))\n",
            "Press [q] to stop, [?] for help\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0musing cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mprofile High, level 3.1\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0m264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00\n",
            "Output #0, mp4, to '/content/munit-interpolation-batch.mp4':\n",
            "  Metadata:\n",
            "    encoder         : Lavf57.83.100\n",
            "    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 720x720, q=-1--1, 24 fps, 12288 tbn, 24 tbc\n",
            "    Metadata:\n",
            "      encoder         : Lavc57.107.100 libx264\n",
            "    Side data:\n",
            "      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1\n",
            "frame=  720 fps= 13 q=-1.0 Lsize=   19398kB time=00:00:29.87 bitrate=5319.2kbits/s speed=0.549x    \n",
            "video:19389kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.048105%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mframe I:4     Avg QP:25.35  size: 74202\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mframe P:201   Avg QP:27.79  size: 40429\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mframe B:515   Avg QP:29.54  size: 22195\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mconsecutive B-frames:  4.4%  0.6%  0.0% 95.0%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mmb I  I16..4:  0.2% 74.1% 25.7%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mmb P  I16..4:  0.1% 12.8%  3.6%  P16..4: 28.3% 34.8% 20.3%  0.0%  0.0%    skip: 0.1%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mmb B  I16..4:  0.0%  1.8%  0.6%  B16..8: 39.6% 31.9% 12.7%  direct: 8.1%  skip: 5.3%  L0:42.0% L1:44.0% BI:14.1%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0m8x8 transform intra:76.3% inter:59.0%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mcoded y,uvDC,uvAC intra: 91.2% 99.1% 94.4% inter: 57.4% 72.6% 37.4%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mi16 v,h,dc,p: 27% 22% 11% 40%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mi8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 10% 19%  9% 12% 10%  9% 10% 10%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mi4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 12% 19%  8% 12% 10%  9%  8%  6%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mi8c dc,h,v,p: 56% 16% 18% 10%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mWeighted P-Frames: Y:60.7% UV:53.2%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mref P L0: 46.3% 17.8% 16.9% 13.8%  5.2%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mref B L0: 79.6% 17.5%  2.8%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mref B L1: 88.8% 11.2%\n",
            "\u001b[1;36m[libx264 @ 0x557f6999be00] \u001b[0mkb/s:5294.30\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZuIynNtN7wMO"
      },
      "source": [
        "# Testing (Alternate)\n",
        "If you’re only using Colab for testing, you’ll need to do the following:\n",
        "1. Move your config .yaml from training into the `configs` folder.\n",
        "2. Create a `models` folder and put the generator .pt file in there (the generator file begins with `gen_`)\n",
        "3. Create an `inputs` folder and a `styles` folder. You’ll put images you want to translate in the `inputs` folder and images you want to guide style in the `styles` folder (See the Guided Translation)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hqj4Ge1NKkJ7"
      },
      "source": [
        "#make some folders (only run once)\n",
        "%mkdir models\n",
        "%mkdir styles"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TDLNbUxV7xkV"
      },
      "source": [
        "#testing with a single image\n",
        "#this generates 10 random images from the model\n",
        "!python test.py --config /content/MUNIT/configs/512-sm_birds2floralmag_folder.yaml --input /content/MUNIT/inputs/birdsAustraliav5Goul_0208.png --output_folder outputs --checkpoint /content/MUNIT/models/gen_00240000.pt --a2b 1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h4rRLZ39V9Tz"
      },
      "source": [
        "!zip -r outputs.zip ./outputs"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nGaL9GJpRbWb"
      },
      "source": [
        "##Testing Options\n",
        "\n",
        "`--num_style`\n",
        "How many \"styles\" you want to output\n",
        "\n",
        "`--num_style_start` What index to start with. If you combine this with `--num_style 1` you can generate only the style you want once determined (for example, run 500 samples, pick your favorite, then only ever test with that one.)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "43-pBsOtaKEs",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "93a43544-126e-483d-ef66-7a621e550ec4"
      },
      "source": [
        "#testing with a single image, use --num_styles to set the number of styles\n",
        "!rm outputs/*.*\n",
        "!python test.py --config /content/MUNIT/configs/512-sm_birds2floralmag_folder.yaml --input /content/MUNIT/inputs/birdsAustraliav5Goul_0208.png --output_folder outputs --checkpoint /content/MUNIT/models/gen_00240000.pt --a2b 1 --num_style 1 --num_style_start 65"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "rm: cannot remove 'outputs/*.*': No such file or directory\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AbQcv9R9941U"
      },
      "source": [
        "##Guided Translation\n",
        "MUNIT also allows you to do what they call \"guided translation.\" This allows you to pass it an image and use that image as the \"style\" (for lack of a better term) for the output image."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZMC-YqZy98-M"
      },
      "source": [
        "#guided translation\n",
        "python test.py --config configs/edges2shoes_folder.yaml --input inputs/edges2shoes_edge.jpg --output_folder results --checkpoint models/edges2shoes.pt --a2b 1 --style inputs/edges2shoes_shoe.jpg"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V44wD5Gq-0Sc"
      },
      "source": [
        "##Batch Testing\n",
        "Both of the above examples only generate samples from one image. If you want to test it on a whole folder of images you use the `test_batch.py` script.\n",
        "\n",
        "Note: you can’t currently batch test with a guided translation image. (I’m working on it.)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Btb7kiV9_0SC"
      },
      "source": [
        "python test_batch.py --config configs/edges2shoes_folder.yaml --input_folder inputs --output_folder outputs --checkpoint models/edges2shoes.pt --a2b 1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GsiVvLuFsGTN"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}