notebooks/demonstration.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d8087d8e",
   "metadata": {},
   "source": [
    "# osms short demo"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4fdc773",
   "metadata": {},
   "source": [
    "## Install osms"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9075cc68",
   "metadata": {},
   "source": [
    "This Jupyter Notebook should be launched in the folder **notebooks**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69716358",
   "metadata": {},
   "source": [
    "We suggest to create a new virtual environment for demonstration.\n",
    "\n",
    "Run the cell below to create a new venv called demo_venv and activate it.\n",
    "\n",
    "The venv allows to isolate all newly installed pakages from system's python interpreter in order not to make confusions with global packages and their versions.\n",
    "\n",
    "You can run the cell below or run these commands in your terminal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9428e417",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pip in /Users/kolya/Documents/Skoltech_PhD/courses/Theoretical Foundations of Data Science/project/OSM-one-shot-multispeaker/.virtualenv/lib/python3.9/site-packages (21.1.1)\r\n"
     ]
    }
   ],
   "source": [
    "!python3 -m venv demo_venv\n",
    "!source demo_venv/bin/activate\n",
    "!python3 -m pip install --upgrade pip"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e28350c",
   "metadata": {},
   "source": [
    "Then one needs to change the current working directory to the root of OSM-one-shot-multispeaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e736b9b8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial working directory: /Users/kolya/Documents/Skoltech_PhD/courses/Theoretical Foundations of Data Science/project/OSM-one-shot-multispeaker/notebooks\n",
      "Current working directory: /Users/kolya/Documents/Skoltech_PhD/courses/Theoretical Foundations of Data Science/project/OSM-one-shot-multispeaker\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "print(f\"Initial working directory: {os.getcwd()}\")\n",
    "os.chdir('../')\n",
    "print(f\"Current working directory: {os.getcwd()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bacfc53",
   "metadata": {},
   "source": [
    "Now one can install osms package"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "29b9144a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing /Users/kolya/Documents/Skoltech_PhD/courses/Theoretical Foundations of Data Science/project/OSM-one-shot-multispeaker\n",
      "\u001b[33m  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.\n",
      "   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.\u001b[0m\n",
      "Requirement already satisfied: numpy in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (1.20.3)\n",
      "Requirement already satisfied: matplotlib in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (3.4.2)\n",
      "Requirement already satisfied: scipy in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (1.6.3)\n",
      "Requirement already satisfied: torch in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (1.8.1)\n",
      "Requirement already satisfied: torchvision in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (0.9.1)\n",
      "Requirement already satisfied: torchtext in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (0.9.1)\n",
      "Requirement already satisfied: torchaudio in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (0.8.1)\n",
      "Requirement already satisfied: ipython in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (7.23.1)\n",
      "Requirement already satisfied: ipywidgets in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (7.6.3)\n",
      "Requirement already satisfied: IProgress in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (0.4)\n",
      "Requirement already satisfied: librosa in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (0.8.0)\n",
      "Requirement already satisfied: SoundFile in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (0.10.3.post1)\n",
      "Requirement already satisfied: inflect in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (5.3.0)\n",
      "Requirement already satisfied: PyYAML in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (5.4.1)\n",
      "Requirement already satisfied: yacs in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (0.1.8)\n",
      "Requirement already satisfied: notebook in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (6.3.0)\n",
      "Requirement already satisfied: unidecode in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (1.2.0)\n",
      "Requirement already satisfied: webrtcvad-wheels in ./.virtualenv/lib/python3.9/site-packages (from osms==1.0.0) (2.0.10.post2)\n",
      "Requirement already satisfied: six in ./.virtualenv/lib/python3.9/site-packages (from IProgress->osms==1.0.0) (1.16.0)\n",
      "Requirement already satisfied: backcall in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (0.2.0)\n",
      "Requirement already satisfied: traitlets>=4.2 in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (5.0.5)\n",
      "Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (3.0.18)\n",
      "Requirement already satisfied: pexpect>4.3 in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (4.8.0)\n",
      "Requirement already satisfied: jedi>=0.16 in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (0.18.0)\n",
      "Requirement already satisfied: pickleshare in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (0.7.5)\n",
      "Requirement already satisfied: decorator in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (5.0.7)\n",
      "Requirement already satisfied: pygments in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (2.9.0)\n",
      "Requirement already satisfied: appnope in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (0.1.2)\n",
      "Requirement already satisfied: matplotlib-inline in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (0.1.2)\n",
      "Requirement already satisfied: setuptools>=18.5 in ./.virtualenv/lib/python3.9/site-packages (from ipython->osms==1.0.0) (56.0.0)\n",
      "Requirement already satisfied: parso<0.9.0,>=0.8.0 in ./.virtualenv/lib/python3.9/site-packages (from jedi>=0.16->ipython->osms==1.0.0) (0.8.2)\n",
      "Requirement already satisfied: ptyprocess>=0.5 in ./.virtualenv/lib/python3.9/site-packages (from pexpect>4.3->ipython->osms==1.0.0) (0.7.0)\n",
      "Requirement already satisfied: wcwidth in ./.virtualenv/lib/python3.9/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython->osms==1.0.0) (0.2.5)\n",
      "Requirement already satisfied: ipython-genutils in ./.virtualenv/lib/python3.9/site-packages (from traitlets>=4.2->ipython->osms==1.0.0) (0.2.0)\n",
      "Requirement already satisfied: nbformat>=4.2.0 in ./.virtualenv/lib/python3.9/site-packages (from ipywidgets->osms==1.0.0) (5.1.3)\n",
      "Requirement already satisfied: jupyterlab-widgets>=1.0.0 in ./.virtualenv/lib/python3.9/site-packages (from ipywidgets->osms==1.0.0) (1.0.0)\n",
      "Requirement already satisfied: widgetsnbextension~=3.5.0 in ./.virtualenv/lib/python3.9/site-packages (from ipywidgets->osms==1.0.0) (3.5.1)\n",
      "Requirement already satisfied: ipykernel>=4.5.1 in ./.virtualenv/lib/python3.9/site-packages (from ipywidgets->osms==1.0.0) (5.5.4)\n",
      "Requirement already satisfied: jupyter-client in ./.virtualenv/lib/python3.9/site-packages (from ipykernel>=4.5.1->ipywidgets->osms==1.0.0) (6.1.12)\n",
      "Requirement already satisfied: tornado>=4.2 in ./.virtualenv/lib/python3.9/site-packages (from ipykernel>=4.5.1->ipywidgets->osms==1.0.0) (6.1)\n",
      "Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in ./.virtualenv/lib/python3.9/site-packages (from nbformat>=4.2.0->ipywidgets->osms==1.0.0) (3.2.0)\n",
      "Requirement already satisfied: jupyter-core in ./.virtualenv/lib/python3.9/site-packages (from nbformat>=4.2.0->ipywidgets->osms==1.0.0) (4.7.1)\n",
      "Requirement already satisfied: attrs>=17.4.0 in ./.virtualenv/lib/python3.9/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets->osms==1.0.0) (21.2.0)\n",
      "Requirement already satisfied: pyrsistent>=0.14.0 in ./.virtualenv/lib/python3.9/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets->osms==1.0.0) (0.17.3)\n",
      "Requirement already satisfied: argon2-cffi in ./.virtualenv/lib/python3.9/site-packages (from notebook->osms==1.0.0) (20.1.0)\n",
      "Requirement already satisfied: terminado>=0.8.3 in ./.virtualenv/lib/python3.9/site-packages (from notebook->osms==1.0.0) (0.9.4)\n",
      "Requirement already satisfied: jinja2 in ./.virtualenv/lib/python3.9/site-packages (from notebook->osms==1.0.0) (2.11.3)\n",
      "Requirement already satisfied: prometheus-client in ./.virtualenv/lib/python3.9/site-packages (from notebook->osms==1.0.0) (0.10.1)\n",
      "Requirement already satisfied: Send2Trash>=1.5.0 in ./.virtualenv/lib/python3.9/site-packages (from notebook->osms==1.0.0) (1.5.0)\n",
      "Requirement already satisfied: nbconvert in ./.virtualenv/lib/python3.9/site-packages (from notebook->osms==1.0.0) (6.0.7)\n",
      "Requirement already satisfied: pyzmq>=17 in ./.virtualenv/lib/python3.9/site-packages (from notebook->osms==1.0.0) (22.0.3)\n",
      "Requirement already satisfied: python-dateutil>=2.1 in ./.virtualenv/lib/python3.9/site-packages (from jupyter-client->ipykernel>=4.5.1->ipywidgets->osms==1.0.0) (2.8.1)\n",
      "Requirement already satisfied: cffi>=1.0.0 in ./.virtualenv/lib/python3.9/site-packages (from argon2-cffi->notebook->osms==1.0.0) (1.14.5)\n",
      "Requirement already satisfied: pycparser in ./.virtualenv/lib/python3.9/site-packages (from cffi>=1.0.0->argon2-cffi->notebook->osms==1.0.0) (2.20)\n",
      "Requirement already satisfied: MarkupSafe>=0.23 in ./.virtualenv/lib/python3.9/site-packages (from jinja2->notebook->osms==1.0.0) (1.1.1)\n",
      "Requirement already satisfied: audioread>=2.0.0 in ./.virtualenv/lib/python3.9/site-packages (from librosa->osms==1.0.0) (2.1.9)\n",
      "Requirement already satisfied: scikit-learn!=0.19.0,>=0.14.0 in ./.virtualenv/lib/python3.9/site-packages (from librosa->osms==1.0.0) (0.24.2)\n",
      "Requirement already satisfied: joblib>=0.14 in ./.virtualenv/lib/python3.9/site-packages (from librosa->osms==1.0.0) (1.0.1)\n",
      "Requirement already satisfied: resampy>=0.2.2 in ./.virtualenv/lib/python3.9/site-packages (from librosa->osms==1.0.0) (0.2.2)\n",
      "Requirement already satisfied: numba>=0.43.0 in ./.virtualenv/lib/python3.9/site-packages (from librosa->osms==1.0.0) (0.53.1)\n",
      "Requirement already satisfied: pooch>=1.0 in ./.virtualenv/lib/python3.9/site-packages (from librosa->osms==1.0.0) (1.3.0)\n",
      "Requirement already satisfied: llvmlite<0.37,>=0.36.0rc1 in ./.virtualenv/lib/python3.9/site-packages (from numba>=0.43.0->librosa->osms==1.0.0) (0.36.0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: packaging in ./.virtualenv/lib/python3.9/site-packages (from pooch>=1.0->librosa->osms==1.0.0) (20.9)\n",
      "Requirement already satisfied: requests in ./.virtualenv/lib/python3.9/site-packages (from pooch>=1.0->librosa->osms==1.0.0) (2.25.1)\n",
      "Requirement already satisfied: appdirs in ./.virtualenv/lib/python3.9/site-packages (from pooch>=1.0->librosa->osms==1.0.0) (1.4.4)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in ./.virtualenv/lib/python3.9/site-packages (from scikit-learn!=0.19.0,>=0.14.0->librosa->osms==1.0.0) (2.1.0)\n",
      "Requirement already satisfied: pillow>=6.2.0 in ./.virtualenv/lib/python3.9/site-packages (from matplotlib->osms==1.0.0) (8.2.0)\n",
      "Requirement already satisfied: cycler>=0.10 in ./.virtualenv/lib/python3.9/site-packages (from matplotlib->osms==1.0.0) (0.10.0)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in ./.virtualenv/lib/python3.9/site-packages (from matplotlib->osms==1.0.0) (1.3.1)\n",
      "Requirement already satisfied: pyparsing>=2.2.1 in ./.virtualenv/lib/python3.9/site-packages (from matplotlib->osms==1.0.0) (2.4.7)\n",
      "Requirement already satisfied: defusedxml in ./.virtualenv/lib/python3.9/site-packages (from nbconvert->notebook->osms==1.0.0) (0.7.1)\n",
      "Requirement already satisfied: pandocfilters>=1.4.1 in ./.virtualenv/lib/python3.9/site-packages (from nbconvert->notebook->osms==1.0.0) (1.4.3)\n",
      "Requirement already satisfied: mistune<2,>=0.8.1 in ./.virtualenv/lib/python3.9/site-packages (from nbconvert->notebook->osms==1.0.0) (0.8.4)\n",
      "Requirement already satisfied: entrypoints>=0.2.2 in ./.virtualenv/lib/python3.9/site-packages (from nbconvert->notebook->osms==1.0.0) (0.3)\n",
      "Requirement already satisfied: nbclient<0.6.0,>=0.5.0 in ./.virtualenv/lib/python3.9/site-packages (from nbconvert->notebook->osms==1.0.0) (0.5.3)\n",
      "Requirement already satisfied: testpath in ./.virtualenv/lib/python3.9/site-packages (from nbconvert->notebook->osms==1.0.0) (0.4.4)\n",
      "Requirement already satisfied: jupyterlab-pygments in ./.virtualenv/lib/python3.9/site-packages (from nbconvert->notebook->osms==1.0.0) (0.1.2)\n",
      "Requirement already satisfied: bleach in ./.virtualenv/lib/python3.9/site-packages (from nbconvert->notebook->osms==1.0.0) (3.3.0)\n",
      "Requirement already satisfied: nest-asyncio in ./.virtualenv/lib/python3.9/site-packages (from nbclient<0.6.0,>=0.5.0->nbconvert->notebook->osms==1.0.0) (1.5.1)\n",
      "Requirement already satisfied: async-generator in ./.virtualenv/lib/python3.9/site-packages (from nbclient<0.6.0,>=0.5.0->nbconvert->notebook->osms==1.0.0) (1.10)\n",
      "Requirement already satisfied: webencodings in ./.virtualenv/lib/python3.9/site-packages (from bleach->nbconvert->notebook->osms==1.0.0) (0.5.1)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.virtualenv/lib/python3.9/site-packages (from requests->pooch>=1.0->librosa->osms==1.0.0) (1.26.4)\n",
      "Requirement already satisfied: idna<3,>=2.5 in ./.virtualenv/lib/python3.9/site-packages (from requests->pooch>=1.0->librosa->osms==1.0.0) (2.10)\n",
      "Requirement already satisfied: chardet<5,>=3.0.2 in ./.virtualenv/lib/python3.9/site-packages (from requests->pooch>=1.0->librosa->osms==1.0.0) (4.0.0)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in ./.virtualenv/lib/python3.9/site-packages (from requests->pooch>=1.0->librosa->osms==1.0.0) (2020.12.5)\n",
      "Requirement already satisfied: typing-extensions in ./.virtualenv/lib/python3.9/site-packages (from torch->osms==1.0.0) (3.10.0.0)\n",
      "Requirement already satisfied: tqdm in ./.virtualenv/lib/python3.9/site-packages (from torchtext->osms==1.0.0) (4.60.0)\n",
      "Using legacy 'setup.py install' for osms, since package 'wheel' is not installed.\n",
      "Installing collected packages: osms\n",
      "  Attempting uninstall: osms\n",
      "    Found existing installation: osms 1.0.0\n",
      "    Uninstalling osms-1.0.0:\n",
      "      Successfully uninstalled osms-1.0.0\n",
      "    Running setup.py install for osms ... \u001b[?25ldone\n",
      "\u001b[?25hSuccessfully installed osms-1.0.0\n"
     ]
    }
   ],
   "source": [
    "!pip3 install ."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1283d9ac",
   "metadata": {},
   "source": [
    "## Speaker Encoder training demo"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9711d07a",
   "metadata": {},
   "source": [
    "This section describes how to manipulate with datasets and how to train an encoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "42e771e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "if not os.path.exists('dataset'):\n",
    "    os.mkdir('dataset')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f17ec98c",
   "metadata": {},
   "source": [
    "Download [LibriSpeech](https://www.openslr.org/resources/12/train-clean-100.tar.gz) dataset and unpack it in the directory `dataset`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1d43032",
   "metadata": {},
   "source": [
    "Import classes and functions for dataset preprocessing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "bf1de8cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from osms.common.configs import get_default_main_configs\n",
    "from osms.tts_modules.encoder.configs import get_default_encoder_config\n",
    "from osms.tts_modules.encoder.data.wav2mel import StandardWav2MelTransform\n",
    "from osms.tts_modules.encoder.data.wav_preprocessing import StandardAudioPreprocessor\n",
    "from osms.tts_modules.encoder.data.dataset import PreprocessLibriSpeechDataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1cf33774",
   "metadata": {},
   "source": [
    "All configs in `osms` are instances of `CfgNode` from `yacs` library. The default values are defined in the corresponding functions inside `osms`. The user can always update these configs from custom \\*.yaml configuration file. However in this demo we won't use customized configurations.\n",
    "\n",
    "`main_configs` contains some general configurations which are available across the whole package.\n",
    "\n",
    "`encoder_config` contains all configurations required for speaker encoder module and its methods and classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "9f398d03",
   "metadata": {},
   "outputs": [],
   "source": [
    "main_configs = get_default_main_configs()\n",
    "encoder_config = get_default_encoder_config()\n",
    "\n",
    "preprocessor = StandardAudioPreprocessor(encoder_config)\n",
    "wav2mel = StandardWav2MelTransform(encoder_config)\n",
    "\n",
    "data_preprocesser = PreprocessLibriSpeechDataset(encoder_config, preprocessor, wav2mel)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efc97351",
   "metadata": {},
   "source": [
    "Before training, the dataset has to be preprocessed. Thus call the method `preprocess_dataset` from `PreprocessLibriSpeechDataset`. \n",
    "\n",
    "In order to speed-up the demonstration, reduce the number of speakers which will be preprocessed by 10. The processing of all speakers requires much time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7de99a20",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 speakers were preprocessed.\n"
     ]
    }
   ],
   "source": [
    "data_preprocesser.preprocess_dataset(n_speakers=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e75bd642",
   "metadata": {},
   "source": [
    "The data from 10 preprocessed speakers can be used for training of Speaker Encoder.\n",
    "\n",
    "Our encoder the is d-vec model.\n",
    "\n",
    "For test purpose we also reduce the number of training steps, just to ensure that training is correct.\n",
    "\n",
    "The whole procedure also requires much time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "3c5d5e39",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from osms.tts_modules.encoder import SpeakerEncoderManager\n",
    "from osms.tts_modules.encoder.data.dataset import SpeakerEncoderDataLoader, SpeakerEncoderDataset\n",
    "from osms.tts_modules.encoder.models import DVecModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e37fb43a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trainable Parameters in dVecModel: 1.424M\n"
     ]
    }
   ],
   "source": [
    "dataset = SpeakerEncoderDataset(encoder_config)\n",
    "dataloader = SpeakerEncoderDataLoader(encoder_config, dataset, 'train')\n",
    "\n",
    "model = DVecModel(encoder_config)\n",
    "\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)\n",
    "\n",
    "encoder_manager = SpeakerEncoderManager(main_configs, model, \n",
    "                                        preprocessor, wav2mel, \n",
    "                                        dataloader, dataloader, \n",
    "                                        optimizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "610512ea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting the training from scratch.\n",
      "Step 1. Train loss value: 4.157451629638672\n",
      "Saving the model (step 2)\n",
      "Step 3. Train loss value: 4.1539411544799805\n",
      "Step 5. Train loss value: 4.152585029602051\n",
      "Saving the model (step 7)\n",
      "Step 7. Train loss value: 4.139642238616943\n",
      "Step 9. Train loss value: 4.148777961730957\n",
      "Stopping Training Session at step #8\n"
     ]
    }
   ],
   "source": [
    "encoder_manager.train_session(number_steps=8, each_n_print_steps=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77d75b80",
   "metadata": {},
   "source": [
    "As one can see the loss decreases constantly. The obtained checkpoints can be found in `train_output/checkpoints` folder."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2626a9d",
   "metadata": {},
   "source": [
    "## One-Shot Learning Example"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e165f703",
   "metadata": {},
   "source": [
    "Run the cell below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "268c8bb2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from osms.common.configs import update_config, get_default_main_configs\n",
    "from osms import MultispeakerManager \n",
    "\n",
    "def mkdir(folder_name):\n",
    "    if not os.path.exists(folder_name):\n",
    "        os.mkdir(folder_name)\n",
    "        print(f'Folder {folder_name} is created!')\n",
    "\n",
    "\n",
    "mkdir('audio_samples')\n",
    "mkdir('texts')\n",
    "mkdir('result_speech')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8e0caec",
   "metadata": {},
   "source": [
    "Create a 5-second .wav file with someone speaking English and put it into the folder **audio_samples**.\n",
    "\n",
    "We suggest to use the app [Audio Recorder](https://apps.apple.com/us/app/audio-recorder-wav-m4a/id1454488895) to record the voice. Set the sample rate to 16HGz there.\n",
    "\n",
    "Create a .txt file with some sentences written in English and put it into the **texts** folder.\n",
    "\n",
    "\n",
    "The examples are already present in these folders. In order to process your wav and txt files, write their names in the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "4b33b052",
   "metadata": {},
   "outputs": [],
   "source": [
    "wav_file_name = 'google_test.wav'\n",
    "# wav_file_name = 'yandex_test.wav'\n",
    "text_file_name = 'test1.txt'\n",
    "output_wav_file_name = 'google_test1.wav'\n",
    "# output_wav_file_name = 'yandex_test1.wav'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "3e6a72b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "main_configs = get_default_main_configs()\n",
    "update_list = ['SPEAKER_SPEECH_PATH', os.path.join('audio_samples', wav_file_name),\n",
    "               'INPUT_TEXTS_PATH', os.path.join('texts', text_file_name),\n",
    "               'OUTPUT_AUDIO_FILE_NAME', output_wav_file_name]\n",
    "main_configs = update_config(main_configs, update_list=update_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5bddaa4",
   "metadata": {},
   "source": [
    "The pretrained weights of models will be automatically downloaded when you create an object of `MultispeakerManager` class first time. These weights will be saved in **checkpoints** folder which will be created in the working directory. The next time you run the programm, the pretrained weights will be loaded from this folder, not downloaded from the Internet.\n",
    "\n",
    "The method `inference()` of `MultispeakerManager` launches the whole multispeaker TTS pipeline. All the steps are done sequentially."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "57499942",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00, ...,\n",
       "       -3.30965176e-08, -1.28919294e-07, -0.00000000e+00])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "multispeaker_manager = MultispeakerManager(main_configs=main_configs)\n",
    "multispeaker_manager.inference()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b64311b",
   "metadata": {},
   "source": [
    "The resulting \\*.wav file will be stored in **result_speech** folder"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}