This program computes automatic pitch correction for vocal performances. It outputs note-wise constant pitch shift values up to 100 cents, equivalent to one semitone. It it can also apply the shifts to the audio.
The program is trained on examples of in-tune singing and applies corrections along a continuous frequency scale.
A pre-trained model is available.
Note: this repository used to be named "autotuner", but has been renamed to "data-driven pitch corrector" to avoid confusion with the proprietary term "Antares Auto-Tune".
Requirements are listed in
requirements.txt and can be installed using
pip install -r requirements.txt.
Data pre-processing requires a program that computes probabilistic YIN (pYIN) pitch analysis. One option is
To run the program on an example included in the repo using a pre-trained model, run:
python rnn.py --extension "3" --resume True --run_training False --run_autotune True
This will output results to directory
results_root_3/realworld_audio_output, which will contain the original
test_mix.wav), and the output of the program (
corrected_mix.wav). Note that the backing track is not
publicly available, so the backing track wav file is silent (a 10-hz sine wave). For this particular example,
the program instead loads the pre-computed constant-Q transform (CQT).
Make sure to first download and uncompress the CQT zip file available at
and place it in
More generally, the program can be run either for training, testing, or automatic pitch correction by setting
the boolean args
In the case of training and testing,
the dataloader takes as input in-tune singing, detunes the notes of the vocal track, and learns to predict the
de-tuning amount. In the case
of automatic pitch correction, the program takes as input real-world performances, predicts corrections for them, and
synthesizes the output.
Multiple other settings and parameters are available in the argument parser in
Define input and output directories along with data format settings in
A pre-trained model is available in the
pytorch_checkpoints_and_models directory. The
README in the
directory provides more details about it.
The program can be run on CPU but runs faster on GPU.
The program requires frame- and note-wise pYIN pitch analyses. Please check directory
./Intonation/realworld_data/pyin for examples of these. The outputs of the Sonic Visualizer are converted from
seconds to frame indices.
Intonation directory needs to contain wav files of the vocals and backing
tracks. The data format should be defined in
intonation.csv should contain a list of
the vocal files and corresponding backing track file names (removing the
.wav extension). Set variables
rnn.py to determine the split between training, testing, and validation data.
The program was trained using the Intonation dataset. More information on the full dataset used for the paper and how to access it via the Stanford DAMP can be found here. Note that the dimension of the backing track CQT in the dataset is different from the one the current program is set to.
Any other data can be used instead.
If you use this code, please refer to the following:
S. Wager, G. Tzanetakis, C. Wang, and M. Kim, "Deep Autotuner: A pitch correcting network for singing performances," in IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), Submitted for publication.
The reference to the dataset creation paper is:
S. Wager, G. Tzanetakis, C. Wang, S. Sullivan, J. Shimmin, M. Kim, and P. Cook, "Intonation: A dataset of quality vocal performances refined by spectral clustering on pitch congruence," in IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), Submitted for publication.
More information on pitch tracking can be found at:
M. Mauch and S. Dixon, “pYIN: A Fundamental Frequency Estimator Using Probabilistic Threshold Distributions,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2014), 2014.