Revisiting Singing Voice Detection : a Quantitative Review and the Future Outlook
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
leglaive_lstm
lehner_randomforest
schluter_cnn
util
.gitattributes
.gitignore
README.md
medley_voice_label.py
medleydb_vocal_songs.txt
requirements.txt
snr_data_gen.py
vibrato_data_gen.py

README.md

Revisiting Singing Voice Detection : a quantitative review and the future outlook

This repo contains code for the paper "Revisiting Singing Voice Detection: a Quantitative Review and the Future Outlook" by Kyungyun Lee, Keunwoo Choi and Juhan Nam at the 19th International Society for Music Information Retrieval Conference (ISMIR) 2018. [pdf, blog post]

Requirements

  • specified in requirements.txt

Public Dataset

  • Jamendo with the same labeling, train/valid/test set split as described in the website.
  • MedleyDB
    We used 61 songs that contain vocals, which can be found in medleydb_vocal_songs.txt.
    Note : MedleyDB does not provide vocal annotations, so we generated labels using the provided instrument activation annotation.
    Download the songs, change path, and run python medley_voice_label.py to generate labels for the 61 songs.

Dataset for stress testing (section 5)

To generate dataset, run

  • python vibrato_data_gen.py for vibrato test in section 5.1.
  • python snr_data_gen.py for SNR test in section 5.2. (Requires modification for path to MedleyDB vocal containing songs.)

Reproduction of singing voice detection models (section 3)

There are 3 reproduced models in the following folders :

  • lehner_randomforest [1]
  • schluter_cnn [2]
  • leglaive_lstm [3]
    Note : Set paths for datasets in each config files within the model folders

Commandline arguments are :

  • --model_name : whatever name you set it during training, and will be saved in ./weights/ folder.
  • --dataset : one of {"jamendo", "vibrato", "snr"}. New dataset can be added with modification in load_data.py (might add RWC pop).

In each model folder, audio processor to preprocess data must be run before playing around with the model.

  • python audio_processor.py --dataset "jamendo" in CNN and RNN model with {"jamendo", "vibrato", "snr"}
  • python vocal_var.py --dataset "jamendo"" in randomforest model with {"jamendo", "vibrato", "snr"}
    Note : This file for randomforest computes vocal variance and concatenates them with the features extracted from the matlab code provided by the authors of [1]. So, this file only provides functions for computing the vocal variance. Either you can add onto this file to compute other features or you can find the matlab code ;)

To train models, run the following in each model folder

  • python main.py --model_name "mynewmodel"

To run pretrained models (models are provided in ./weights/ folder), run the following in each model folder

  • python test.py --model_name "mynewmodel" --dataset "jamendo"

References

  • [1] Bernhard Lehner, Gerhard Widmer, and Reinhard Sonnleitner. "On the reduction of false positives in singing voice detection." pdf
  • [2] Jan Schlueter and Thomas Grill. "Exploring data augmentation for improved singing voice detection with neural networks." pdf
  • [3] Simon Leglaive, Romain Hennequin, and Roland Badeau. "Singing voice detection with deep recurrent neural network." pdf

TO DO (2018.06)

  • Upload notebook file for model analysis and audacity compatible label generation.