Turkish-TTS

Turkish Speech Synthesis Project based on Tacotron2

Tacotron 2 (without WaveNET and HiFi-GAN)

Turkish implementation code of [A Novel End-to-End Turkish Text-to-Speech (TTS) System via Deep Learning] (MDPI, electronics).

https://www.mdpi.com/2079-9292/12/8/1900

Cite

Oyucu, S. A Novel End-to-End Turkish Text-to-Speech (TTS) System via Deep Learning. Electronics 2023, 12, 1900. https://doi.org/10.3390/electronics12081900

Turkish TTS implementation data of [Preparing A Balanced Corpus for Development of Turkish Speech Synthesis Systems]

https://dergipark.org.tr/en/pub/gazibtd/issue/79101/1159289

Cite

Cücen, M. S. , Oyucu, S. & Polat, H. (2023). Türkçe Konuşma Sentezleme Sistemlerinin Geliştirilmesi için Dengeli Bir Veri Kümesi Hazırlama . Bilişim Teknolojileri Dergisi , 16 (3) , 237-249 . DOI: 10.17671/gazibtd.1159289

This implementation includes distributed and automatic mixed precision support and uses the Turkish TTS dataset.

Distributed and Automatic Mixed Precision support relies on NVIDIA's Apex and AMP.

Visit our link for audio samples using our Tacotron 2 and HiFi-GAN models.

Pre-requisites

1. NVIDIA GPU + CUDA cuDNN

Setup

1. Download and extract the Turkish TTS Corpus
2. Clone this repo: git clone https://github.com/NVIDIA/tacotron2.git
3. CD into this repo: cd tacotron2
4. Initialize submodule: git submodule init; git submodule update
5. Update .wav paths: sed -i -- 's,DUMMY,dataset_folder/wavs,g' filelists/*.txt
   • Alternatively, set load_mel_from_disk=True in hparams.py and update mel-spectrogram paths
6. Install PyTorch 1.0
7. Install Apex
8. Install python requirements or build docker image
   • Install python requirements: pip install -r requirements.txt

Symbols

Symbols are set for Turkish.

Turkish Abbreviations

Turkish Abbreviations Txt File

Numbers

Numbers are set for Turkish.

Cleaners

Cleaner sare set for Turkish. (deepzeka_cleaners)

Training

1. python train.py --output_directory=outdir --log_directory=logdir
2. (OPTIONAL) tensorboard --logdir=outdir/logdir

Training using a pre-trained model

Training using a pre-trained model can lead to faster convergence
By default, the dataset dependent text embedding layers are ignored

1. python train.py --output_directory=outdir --log_directory=logdir -c tacotron2_statedict.pt --warm_start

Multi-GPU (distributed) and Automatic Mixed Precision Training

1. python -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True

N.b. When performing Mel-Spectrogram to Audio synthesis, make sure Tacotron 2 and the Mel decoder were trained on the same mel-spectrogram representation.

Tacotron2 and HiFi-GAN Inference Notebook

Tacotron2 and HiFi-GAN Inference Notebook

Acknowledgements

This implementation uses code from the following repos: Nvidia, Keith Ito, Prem Seetharaman as described in our code.

We are inspired by Ryuchi Yamamoto's Tacotron PyTorch implementation.

We are thankful to the Tacotron 2 paper authors, specially Jonathan Shen, Yuxuan Wang and Zongheng Yang.