Hackathon

Let's red team MMS and Whisper!

Instructions

First, clone this repo locally (or on the cluster.)

cd /path/where/to/save/this/repo  # e.g. in Mac: cd /Users/myself/Documents
git clone https://github.com/angelo-ortiz/hackathon.git

Then, install the requirements for MMS

cd hackathon
conda env create -f environment.yml
conda activate hack

The final requirement is fairseq. To install it, do the following

cd /path/where/to/save/fairseq  # e.g. in Mac: cd /Users/myself/Documents
git clone https://github.com/pytorch/fairseq
cd fairseq
pip install --editable ./
cd ..

Then, install the model

wget -P ./models_new 'https://dl.fbaipublicfiles.com/mms/asr/mms1b_all.pt'

Then create a sample audio file (or record one yourself, call it audio.wav (16khz, please) and put it in the directory ./audio_samples)

mkdir ./audio_samples
say -o audio_samples/audio "Hello, how are you?"
sox audio_samples/audio.aiff audio_samples/audio.wav

Before running the model, modify the following line in /path/to/fairseq/examples/mms/asr/config/infer_common.yaml. More importantly, prepend your home directory (e.g. in Mac: /Users/{env:USER})

dir: /Users/${env:USER}/checkpoint/${env:USER}/${env:PREFIX}/${common_eval.results_path}

If you do not have a GPU, you also need to add the following lines to the same file:

common:
  cpu: true

Finally, you can run the model in the notebook asr_with_mms.ipynb

Links

• MMS 1B model
• ASR example