RAVE_Training_Template--Colab.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RAVE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xRbFd2IqdP3Q"
   },
   "source": [
    "**RAVE** is a variational autoencoder for fast and high-quality neural audio synthesis by Antoine Caillon and Philippe Esling.\n",
    "\\\n",
    "[Article on arxiv](https://arxiv.org/abs/2111.05011) & [Source code on Github](https://github.com/acids-ircam/RAVE)\n",
    "\n",
    "---\n",
    "<small>*This notebook is based on the one from [Hexorcismos](https://colab.research.google.com/drive/1ih-gv1iHEZNuGhHPvCHrleLNXvooQMvI?usp=sharing) and comes with updates, additional hints and comments.\n",
    "\\\n",
    "Note that this notebook currently supports training RAVE until version 2.2.2, support for version 2.3 and beyond will be added soon.* \n",
    "\\\n",
    "\\\n",
    "Notebook Author: [Martin Heinze](https://github.com/devstermarts)\\\n",
    "Date: 17.01.2023"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZwzNM9kGTijj"
   },
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "_Rlc6DI7_Epj"
   },
   "outputs": [],
   "source": [
    "#@title **Colab setup** (run this each time you want to start or resume training)\n",
    "#@markdown **Select the version of RAVE you want to use.** \\ \n",
    "#@markdown *It's recommended to pin this notebook to a version/ release tag of RAVE, to avoid errors when resuming training. Note that this notebook currently supports training RAVE until version 2.2.2, support for version 2.3 and beyond will be added soon.* https://github.com/acids-ircam/RAVE/tags\n",
    "version = \"2.2.2\" #@param {type:\"string\"}\n",
    "\n",
    "#Install Conda\n",
    "!mkdir /content/download\n",
    "%cd /content/download\n",
    "!curl -L https://repo.anaconda.com/miniconda/Miniconda3-py39_4.12.0-Linux-x86_64.sh -o miniconda.sh\n",
    "!chmod +x miniconda.sh\n",
    "!sh miniconda.sh -b -p /content/miniconda\n",
    "\n",
    "#Install RAVE\n",
    "!/content/miniconda/bin/pip install acids-rave==$version\n",
    "\n",
    "#Upgrade ipython ipykernel, install ffmpeg\n",
    "!/content/miniconda/bin/pip install --upgrade ipython ipykernel\n",
    "!/content/miniconda/bin/conda install ffmpeg --yes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "QzdJDIy-Qyqz"
   },
   "outputs": [],
   "source": [
    "#@title **Mount Google Drive**\n",
    "\n",
    "from google.colab import drive\n",
    "drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aDl9A4GeTy7I"
   },
   "source": [
    "## Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "v_uNW4HZg4f0"
   },
   "outputs": [],
   "source": [
    "#@title **Define variables** (run this each time or when you change something)\n",
    "\n",
    "#@markdown **1. Add the path to your working folder**\\\n",
    "#@markdown *This should be the main folder for all the generated data from pre processing and training. Make sure this is on your Google Drive, which is mounted to /content/drive/MyDrive* \n",
    "yourfolder = \"/content/drive/MyDrive/myfolder/\" #@param {type:\"string\"}\n",
    "#@markdown ---\n",
    "#@markdown **2. Add the path to your training dataset** *- aka the audio files you want RAVE to train on*\n",
    "dataset = \"/content/drive/MyDrive/myfolder/dataset/\" #@param {type:\"string\"}\n",
    "#@markdown ---\n",
    "#@markdown **3. Add the path to where the metadata from pre processing should be stored**\\ \n",
    "#@markdown *This should create a lock.mdb, data.mdb and metadata.yaml file in your folder.*\n",
    "processed = \"/content/drive/MyDrive/myfolder/meta/\" #@param {type:\"string\"}\n",
    "#@markdown ---\n",
    "#@markdown **4. Give your training a name**\\\n",
    "#@markdown *Avoid blanks/ spaces. A subfolder is created in your main folder with your training's name and a string suffix. Your model data and checkpoints will be stored inside this folder.*\n",
    "name = \"training\" #@param {type:\"string\"}\n",
    "#@markdown ---\n",
    "#@markdown **5. Select configuration parameters.** *Hint: there are various other configurations available. Use 'rave train --helpfull' to get a complete overview.*\n",
    "architecture = \"v3.gin\" #@param [\"v3.gin\", \"v2.gin\", \"v1.gin\", \"discrete.gin\", \"onnx.gin\", \"raspberry.gin\"]\n",
    "regularization = \"wasserstein\" #@param [\"wasserstein\", \"spherical\"] \n",
    "#@markdown Note that regularization only works with v2.gin, if you want to train with any of the other architectures, the second --config parameter needs to be manually deleted in the training section below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "iZ6ECduhA2_Q"
   },
   "outputs": [],
   "source": [
    "#@title **Pre process dataset** (only needed once per dataset)\n",
    "\n",
    "!mkdir $processed\n",
    "!/content/miniconda/bin/rave preprocess \\\n",
    "--input_path $dataset \\\n",
    "--output_path $processed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "B81jNhVUAKJ0"
   },
   "outputs": [],
   "source": [
    "#@title **Load Tensorboard for monitoring training** (each training)\n",
    "#@markdown **Tensorboard will be loaded from inside your working folder.**\\\n",
    "#@markdown *Works best with Chrome. Disable ad and script blockers. Some monitoring data is only shown in the second phase of the training.* \n",
    "\n",
    "%cd $yourfolder\n",
    "%load_ext tensorboard\n",
    "%tensorboard --logdir . "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yO4kw0XKT4zx"
   },
   "source": [
    "## Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "s3ne3LYOan78"
   },
   "outputs": [],
   "source": [
    "#@title **Initial training** (start with this)\n",
    "%cd $yourfolder\n",
    "\n",
    "#Initiate training. Note that you can change various training parameters by using --override\n",
    "!/content/miniconda/bin/rave train \\\n",
    "--config $architecture \\\n",
    "--config $regularization \\\n",
    "--db_path $processed \\\n",
    "--name $name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "_D4E5lZaOpE6"
   },
   "outputs": [],
   "source": [
    "#@title **Resume training** (continue with this)\n",
    "#@markdown **Add the path to the most recent checkpoint folder in order to resume training.**\\ \n",
    "#@markdown *These can be found in the /runs subfolder. Training name and parameters are taken from above section's settings. Make sure to run that section before resuming.*\n",
    "checkpoint = \"/content/drive/MyDrive/your_folder/runs/your_training_name_with_a_random_string_in_the_end/version_X/checkpoints/\" #@param {type:\"string\"}\n",
    "\n",
    "%cd $yourfolder\n",
    "\n",
    "#Resume training\n",
    "!/content/miniconda/bin/rave train \\\n",
    "--config $architecture \\\n",
    "--config $regularization \\\n",
    "--db_path $processed \\\n",
    "--name $name \\\n",
    "--ckpt $checkpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6qIsXQV8T9Rg"
   },
   "source": [
    "## Export"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "kcgmc-B0K9Fv"
   },
   "outputs": [],
   "source": [
    "#@title **Export model**\n",
    "#@markdown **Add the path to your model data.**\\ \n",
    "#@markdown *This is a subfolder in /runs with your training name and a string suffix.* \n",
    "modelfolder = \"/content/drive/MyDrive/your_project_folder/runs/your_training_name_with_a_random_string_in_the_end/\" #@param {type:\"string\"}\n",
    "\n",
    "!/content/miniconda/bin/rave export \\\n",
    "--run $modelfolder \\\n",
    "--streaming \\\n",
    "--stereo"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [
    "ZwzNM9kGTijj",
    "aDl9A4GeTy7I",
    "yO4kw0XKT4zx",
    "6qIsXQV8T9Rg"
   ],
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}