I aim to create almost natural-sounding and understandable synthetic voices based on the principle of sound concatenation from an automatically pre-built speaker database. It should be as easy as possible for a user to make new models and use them immediately, without major technical overhead.
An example of generated speech is included in this repository, as well as basic files to generate a model based on my voice.
Dependencies and installation
Sinatoki depends on several other open source projects, namely
- composer: fetch other php dependencies
- docker: required to run the container of the forced aligner gentle
- gentle: aligning audio with transcript on phoneme level (is automatically downloaded and installed when the model creation script creates the docker container)
- espeak-ng: handles correct pronunciation and emphasis
- ffmpeg: slices speech
- mp3 codecs provided by your distribution
Make sure you have installed them on your system before proceeding.
# Ubuntu only, getting the dependencies sudo apt install php php-cli composer docker.io espeak ffmpeg git # Cross-platform installation git clone https://github.com/nkreer/sinatoki cd sinatoki composer install # Creating a demo model from included files # You have to start the docker daemon first sudo dockerd # Then run the included script # This requires root privileges because it interacts with the container # Note that the tool only supports .mp3 as base file format for now sudo ./create-voicemodel.sh models/niklas/base niklas-baseaudio.mp3 niklas-transcript.txt niklas # Generate example speech (vlc required) php synthesizer.php niklas Hello World vlc output.mp3
If you'd like to hack & contribute to the project, you'll be happy to find well-commented and descriptive source code. I look forward to see your pull requests!
Please note that generating voice models will take a significant amount of time right now, depending on your hardware and the length of your prepared recording.
Ethically, using this project shouldn't be much of a concern as you can clearly hear that sinatoki voices sound very robotic and not natural. The primary use of this tool is in entertainment and education - not so much in actual real-world scenarios.
Although we can create understandable speech at the moment, some more things will have to be done to make it sound even more natural:
- Save the same context (and phoneme) multiple times, but include context as to where it has been used (surrounding words, etc.)
- do not ignore emphasis
- Ignore unusable slices (silence, too short, etc.)
- Remove glottal stops automatically
- implement interface so the system/a browser can use the voices
Feel free to work on either of these and contribute your changes back to the project!
"sina toki" is Toki Pona for "your voice", expressing all the aims of this project in a simple phrase. Ona pona?
Sinatoki is an improved version of the speech synthesizer originally created at Jugend hackt Berlin 2019, using the same technologies but with more intelligent processing of the available data.