Thai TTS Tacotron

Thai TTS Tacotron is the text to speech model in Thai trained by Tacotron2.

Model from https://github.com/NVIDIA/tacotron2, the pytorch implementation of Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions and training with TSynC2_Nun Dataset.

Installation

Use the git clone to clone the tacotron project.

%tensorflow_version 1.x
import os
from os.path import exists, join, basename, splitext
git_repo_url = 'https://github.com/Prim9000/tacotron2.git'
project_name = splitext(basename(git_repo_url))[0]
if not exists(project_name):
  # clone and install
  !git clone -q --recursive {git_repo_url}
  !cd {project_name}/waveglow && git checkout 9168aea
  !pip install -q librosa unidecode
  
import sys
sys.path.append(join(project_name, 'waveglow/'))
sys.path.append(project_name)
import time
import matplotlib
import matplotlib.pylab as plt
plt.rcParams["axes.grid"] = False

Training

Parameters tuning and recheck your training files:

1. train.py --output_directory=outdir --log_directory=logdir
2. (OPTIONAL) tensorboard --logdir=outdir/logdir

Training using pre-trained (warm-start)

Training using a pre-trained model can lead to faster convergence

By default, the dataset dependent text embedding layers are ignored.

python train.py --output_directory=outdir --log_directory=logdir -c tacotron2_statedict.pt --warm_start

Synthesizing Text

checkpoint_path = "checkpoint_path/checkpoint" # your model's checkpoint path
model = load_model(hparams)
model.load_state_dict(torch.load(checkpoint_path)['state_dict'])
_ = model.cuda().eval().half()

text = "ยินดีที่ได้รู้จัก นี่คือเสียงจากปัญญาประดิษฐ์" #change input text here

sequence = np.array(text_to_sequence(text, ['english_cleaners']))[None, :]
sequence = torch.autograd.Variable(
    torch.from_numpy(sequence)).cuda().long()
    
mel_outputs, mel_outputs_postnet, _, alignments = model.inference(sequence)
plot_data((mel_outputs.float().data.cpu().numpy()[0],
           mel_outputs_postnet.float().data.cpu().numpy()[0],
           alignments.float().data.cpu().numpy()[0].T))
           
with torch.no_grad():
    audio = waveglow.infer(mel_outputs_postnet, sigma=0.666)
ipd.Audio(audio[0].data.cpu().numpy(), rate=hparams.sampling_rate)

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

References and Related repos

WaveGlow Faster than real time Flow-based Generative Network for Speech Synthesis

nv-wavenet Faster than real time WaveNet.

Tacotron 2, the official repository implementation with Pytorch.

Medium

Lifelike Speech Synthesis | Thai Text To Speech with Tacotron2