Adding Speech Models
Speech recognition uses large databases of information for doing the actual recognition, these are called models. To set up recognition in SpeechToLife we need to add the recognition models for the languages we would like to support. This document describes this process:
First, we grab the models we want to use in our project from here. There are many models to choose from but I suggest you use the lightweight versions as the large versions require too high of a memory footprint. Let's grab vosk-model-small-en-us-0.15 and vosk-model-small-es-0.22 for English and Spanish.
In our content directory folder we need to create a folder called 'SpeechToLife' and in that folder create another folder called 'vosk' and in that folder create another folder called 'models' and in this 'models' folder, we can add the models we have downloaded from the Vosk website. Here is the directory structure from our project, 'Content/SpeechToLife/vosk/models' where we have two folders for our models we have downloaded.
In the English models zip there is a folder called 'vosk-model-small-en-us-0.15', which we place in the models directory and rename to 'en-us' for simplicity. We do the same thing for the Spanish language model naming it 'es'. Here are the folders with their model contents:
In the Unreal Editor open Project Settings and on the side bar head down to "Speech To Life" and open it. Inside there are a couple of properties:
- DefaultLocale - You can set this to a default locale to get ready when the application starts. It is better to call SetLocale in blueprint to the locale of the users local machine instead of having a default locale set, but this is here in any case.
- Locales - This is where you will add the new models we just added to the content folder.
Each locale in Locales is defined by these properties:
- Locale - This is the reference name for the locale. This is the value you would pass into SetLocale.
- Model Folder Name - This is the name of the folder for this locales model relative to 'Content/SpeechToLife/vosk/models'.
Add two new locales to the locales array using the + button.
In the first locale set 'Locale' to 'en' for English. Set the 'Model Folder Name' to 'en-us' which is the name we gave the English model folder.
In the second locale set 'Locale' to 'es' for Spanish. Set the 'Model Folder Name' to 'es' which is the name we gave the Spanish model folder.
It should look like this:
And the content browser should show the two models we added earlier:
This is setup exactly as it can be found in the Example Project here
Open project settings and under Project->Packaging add 'SpeechToLife' to the 'Additional Non-Asset Directories to Copy' array like so:
And that is it! You have added English and Spanish recognition to your project. You can add other languages in the same way. In blueprint call SetLocale from the SpeechToLifeSubsystem:
