Audio keyword spotting on Sitara MPU

This repository has been validated on Texas Insruments AM62x and AM62Ax microprocessors for the 8.6 SDK.

What is this repo?

Keyword spotting allows automatic recognition of speech words within a limited vocabulary to assist human-machine interaction. This task often uses machine learning models like neural networks, and can run in a limited memory and CPU processing footprint on a microcontroller or microprocessor.

This repo hosts python3 code for audio keyword spotting using two (tinyml-kws and matchboxnet) separate models trained on the Google Speech commands dataset (v1 and v2, respectively) running on the Am62x/AM62Ax SoC (aka 'the device') with the Linux SDK. It has been validated on these devices, but will likely run on other linux-based devices since there are very few hardware restrictions.

What is required

• An AM62 or AM62A starter kit EVM
• SD card
• USB to USB-micro cable
• ethernet network connection
• USB microphone

How to run the demo

1. Setup the SDK on an SD card for the starter kit EVM's selected device/SoC according to the respective device's user guide (AM62x and AM62Ax. Follow that guide until a linux terminal session is available through serial/USB or internet/SSH
2. Clone this repository onto the device. This may require setting proxy variables like HTTPS_PROXY if the device is behind a firewall
3. Run the 'install_on_evm.sh' script. This will download and build the portaudio dependency, and then setup a few python libraries for audio processing
4. "Source" the 'quick_setup.sh' script to set an environment variable that is needed to load pyaudio in python: source ./quick_setup.sh
5. Within the two subfolders, run either of the two python scripts starting with 'audio-inference'. Note that you will need to have the USB microphone plugged in and know the device index within linux. This is 1 by default, but the 'detect_microphone.py' script will also help identify them.

Words recognized

The tinyml model can recognize 12 words: Down, Go, Left, No, Off, On, Right, Stop, Up, Yes, Silence, Unknown

The matchboxnet model is capable of recognizing a larger set of words: visual, wow, learn, backward, dog, two, left, happy, nine, go, up, bed, stop, one, zero, tree, seven, on, four, bird, right, eight, no, six, forward, house, marvin, sheila, five, off, three, down, cat, follow, yes

Support and Resources

Please direct questions to the Processors e2e

• Main TI Edge AI page: https://ti.com/edgeai
• TI Arm Processors
• ML Commons Tiny repo
• Nemo keyword spotting models using Matchboxnet