HpRNet : Incorporating Residual Noise Modeling for Violin in a Variational Parametric Synthesizer

Krishna Subramani, Preeti Rao : IIT Bombay

---

This repository contains the code for HpRNet, an extension of our previous work VaPar Synth. It is a Conditional Variational Autoencoder trained on a source-filter inspired parametric representation. However we focus on the generative modeling of the residual bow noise to make for more natural tone quality.

We also introduce a new dataset, the Carnatic Violin Dataset

Setting up an Anaconda Environment

For the necessary libraries/prerequisites, please use conda/anaconda to create an environment (from the environment.yml file in this repository) with the command

conda env create -f environment.yml

Also install SMS-Tools in the same environment. With these, all the code in the repository can be run inside this environment by activating it.

Code

A lot of our code is recycled and modified from our previous project VaPar Synth.

1. Dependencies: Functions for TAE extraction, PyTorch Dataloading, Sampling from the network etc.
2. Parametric: Obtaining the parametric representation of the audio.
3. Network: PyTorch code for the various networks.
4. Analysis: Code to analyze the network outputs (compute MSE/visualize Latent Space with t-SNE)
5. GUI: We also present a simple GUI (inspired from SMS-Tools!) for researchers to play around with. They can load pre-trained network weights and reconstruct/generate user input audio files.

---

If you use the dataset or the code, please refer to our work as:

@dataset{krishna_subramani_2020_3940330,
  author       = {Krishna Subramani and
                  Preeti Rao},
  title        = {Carnatic Violin Dataset},
  month        = jul,
  year         = 2020,
  publisher    = {Zenodo},
  version      = {1.0},
  doi          = {10.5281/zenodo.3940330},
  url          = {https://doi.org/10.5281/zenodo.3940330}
}