examples/HOWTO_run_Kandinsky_with_Ikomia_API.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9XIBGlEafDQf"
      },
      "source": [
        "<img src='https://github.com/Ikomia-dev/notebooks/blob/main/examples/img/banner_ikomia.png?raw=true'>\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DvouGVeYfDQg"
      },
      "source": [
        "#  Create your image with Kandinsky 2.2\n",
        "\n",
        "\n",
        "**Kandinsky 2.2** is a text-conditional diffusion model based on unCLIP and latent diffusion. This model series, developed by a team from Russia, has evolved through several iterations, each bringing new features and improvements in image synthesis from text descriptions.\n",
        "\n",
        "\n",
        "![illustration kandinsky](https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinskyv22/%20blue%20eyes.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-STLXz8ifDQh"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Please use a GPU for this tutorial.\n",
        "\n",
        "In the menu, select \"Runtime\" then \"Change runtime type\", choose GPU in \"Hardware accelerator\".\n",
        "\n",
        "Check your GPU with the following command:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "!nvidia-smi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cV0_2S0SfDQh"
      },
      "source": [
        "You need to install Ikomia Python API with pip\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "cbvRlv_ufDQh",
        "outputId": "e3893478-603b-467b-96a3-b272121649b3"
      },
      "outputs": [],
      "source": [
        "!pip install ikomia"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-j3VbsAYfDQi"
      },
      "source": [
        "---\n",
        "\n",
        "**-Google Colab ONLY- Restart runtime**\n",
        "\n",
        "Click on the \"RESTART RUNTIME\" button at the end the previous window.\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Run Kandinsky 2.2 text2img"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "uBp98pWxiHXq"
      },
      "outputs": [],
      "source": [
        "from ikomia.dataprocess.workflow import Workflow\n",
        "\n",
        "\n",
        "# Init your workflow\n",
        "wf = Workflow()\n",
        "\n",
        "# Add algorithm\n",
        "algo = wf.add_task(name = \"infer_kandinsky_2\", auto_connect=False)\n",
        "\n",
        "# Edit paramerters\n",
        "algo.set_parameters({\n",
        "    'model_name': 'kandinsky-community/kandinsky-2-2-decoder',\n",
        "    'prompt': 'A Woman Jedi fighter performs a beautiful move with one lightsabre, full body, dark galaxy background, look at camera, Ancient Chinese style, cinematic, 4K.',\n",
        "    'negative_prompt': 'low quality, bad quality',\n",
        "    'prior_num_inference_steps': '25',\n",
        "    'prior_guidance_scale': '4.0',\n",
        "    'num_inference_steps': '100',\n",
        "    'guidance_scale': '1.0',\n",
        "    'seed': '-1',\n",
        "    'width': '1280',\n",
        "    'height': '768',\n",
        "    })\n",
        "\n",
        "\n",
        "# Generate your image\n",
        "wf.run()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from ikomia.utils.displayIO import display\n",
        "\n",
        "from PIL import ImageShow\n",
        "ImageShow.register(ImageShow.IPythonViewer(), 0)\n",
        "\n",
        "# Display the image\n",
        "display(algo.get_output(0).get_image())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### List of parameters\n",
        "\n",
        "- **model_name** (str) - default 'kandinsky-community/kandinsky-2-2-decoder': Name of the latent diffusion model. \n",
        "- **prompt** (str) - default 'portrait of a young women, blue eyes, cinematic' : Text prompt to guide the image generation .\n",
        "- **negative_prompt** (str, *optional*) - default 'low quality, bad quality': The prompt not to guide the image generation. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`).\n",
        "- **prior_num_inference_steps** (int) - default '25': Number of denoising steps of the prior model (CLIP).\n",
        "- **prior_guidance_scale** (float) - default '4.0':  Higher guidance scale encourages to generate images that are closely linked to the text prompt, usually at the expense of lower image quality. (minimum: 1; maximum: 20).\n",
        "- **num_inference_steps** (int) - default '100': The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.\n",
        "- **guidance_scale** (float) - default '1.0':  Higher guidance scale encourages to generate images that are closely linked to the text prompt, usually at the expense of lower image quality. (minimum: 1; maximum: 20).\n",
        "- **height** (int) - default '768: The height in pixels of the generated image.\n",
        "- **width** (int) - default '768: The width in pixels of the generated image.\n",
        "- **seed** (int) - default '-1': Seed value. '-1' generates a random number between 0 and 191965535.\n",
        "\n",
        "\n",
        "*note:\"prior model\" interprets and encodes the input text to understand the desired image content, while the \"decoder model\" translates this encoded information into the actual visual representation, effectively generating the image based on the text description.*"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "V100",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "venvapi",
      "language": "python",
      "name": "venvapi"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    },
    "orig_nbformat": 4
  },
  "nbformat": 4,
  "nbformat_minor": 0
}