If you use VocalMat4Rats or any part of it in your own work, please cite: Scott et al. 2024, and Fonseca et al:
@article{Fonseca2021AnalysisOU,
title={Analysis of ultrasonic vocalizations from mice using computer vision and machine learning},
author={Antonio H. O. Fonseca and Gustavo Madeira Santana and Gabriela M Bosque Ortiz and Sergio Bampi and Marcelo O. Dietrich},
journal={eLife},
year={2021},
volume={10}
}
For more information, visit the VocalMat team's website: VocalMat - Dietrich Lab
Dataset and audios used in the original VocalMat paper are available at: OSF Repository
Dataset for VocalMat4Rats will be available upon publication.
VocalMat4Rats is an automated tool that identifies and classifies 50-kHz range rat vocalizations.
VocalMat is divided into two main components. The VocalMat Identifier, and the VocalMat Classifier.
VocalMat Identifier detects vocalization candidates in the audio file. Vocalization candidates are detected through a series of image processing operations and differential geometry analysis over spectrogram information. The VocalMat Identifier outputs (optional) a MATLAB formatted file (.MAT) with information about the spectral content of detected vocalizations (e.g., frequency, intensity, timestamp), that is later used by the VocalMat Classifier.
VocalMat Classifier uses a Convolutional Neural Network (CNN) to classify each short vocalization candidate into 12 distinct labels: Complex, Complex trill, Downward ramp, Flat, Inverted-U, Short, Split, Step down, Step up, Trill, Upward ramp, and noise.
- 11 Classification Classes: VocalMat is able to distinguish between 11 classes of vocalizations (see Figure above), according to adapted definitions from Wright et al. (2010).
- Noise Detection: eliminates vocalization candidates associated to mechanical or segmentation noise.
- Harmonic Detection: detects vocalizations with components overlapping in time.
- Fast Performance: optimized versions for personal computers and high-performance computing (clusters)
- Signal detection changes: Increased time of absent signal for VM to determine the following signal to be a new USV, to account for species differences in vocalizations.
- Minimum frequency threshold: At ~line 140 in VM identifier routine, we've set a minimum threshold at 30kHz to exclude 22-kHz aversives, this can be modified at your leisure
- Wright et al. 2010 style classifiers: VM4Rats Uses our classification scheme for rat USVs, if you wish to train a new model using our classification scheme, see the training routine train_model_Rat
- Note: Clustering and diffusion maps are currently disabled for this version of VocalMat4Rats as these were throwing errors.
- Synthetic training data creation routine: Create your own synthetic training data images with our USV_morpher.m routine, which takes as input images created through VocalMat. Note that you need to check these images for likeness before including them in your training set.
You must have Git LFS installed to be able to fully clone the repository. Download Git LFS
If in doubt, proceed to the Manual Download section
$ git clone https://github.com/scoki211/VocalMat4Rats.git$ git clone -b VocalMat_RC --single-branch https://github.com/scoki211/VocalMat4Rats.gitYou can use a Git client to clone our repository, we recommend GitHub's own client:
Download at: https://desktop.github.com
You can download VocalMat by using GitHub's Download Zip option. However, since we use Git LFS (Git Large File Storage), a necessary file will not be downloaded
Download this repository as a zip file: Download Zip
Extract the .zip file. This is the VocalMat directory.
Download the neural network model file: Download Model
Place the model file in the vocalmat_classifier folder inside the VocalMat directory.
- vocalmat_identifier: Directory with all files and scripts related to the VocalMat Identifier
- vocalmat_classifier: Directory with all files and scripts related to the VocalMat Classifier
- audios: Directory with audio files you want to analyze in the
audiosdirectory
Navigate to the VocalMat directory in MATLAB and run VocalMat.m by either opening the file or typing VocalMat in MATLAB's command window. Once VocalMat is running, choose the audio file you want to analyze. An example audio file is provided, and you can use it to test VocalMat.
VocalMat outputs a directory with the same name as the audio file that was analyzed. Inside that directory there will be two directories (All, and All_axes if save_plot_spectrograms=1), and two Microsoft Excel (.xlsx) files. Inside All_axes you will find one image for each vocalization candidate detetcted with the resulting segmentation illusrated by blue circles. The raw original images are available inside All. The main Excel file has the same name of the audio file analyzed (audio_file_name.xlsx). This file contains information on each vocalization, such as start and end time, duration, frequency (minimum, mean and maximum), bandwidth, intensity (minimum, mean, maximum and corrected based on the backgroun), existence of harmonic components or distortions (noisy), and call type. The second excel file named as audio_file_name_DL.xlsx shows the probability distribution for each vocalization candidate for the different vocal classes.
While these data that were reported in Scott et al. 2024 were suitable for our needs under low noise conditions, it should be noted that differences in recording conditions, microphone gain and recording paradigm may lead to different levels of performance. Thus, training of your own network (utilizing images created through VocalMat audio processing) may be a desirable option. You can access the training routine through VocalMat4Rats-master > vocalmat_classifier > training > train_model_Rat. You will see at line 21 you will need to specify the folder where your images are located. From line 75 you will need to specify your desired classification schema, if you wish to change it from the present one. The folder where your images are located should have a subfolder for each classification category specified, within which is contained your training images. If adjusting classification schema, you will also need to adjust the routine vocalmat_classifier.m before it will run successfully, to do so you will need to adjust classifiers from line 52 and again from line 348. Included in this repo is now the USV_morpher.m routine, which allows you to create synthetic training data from a VocalMat USV image (found in the 'all' folder from the VocalMat output).
We have included a frequency cut-off at 30-kHz as standard, but you may wish to modify this to suite your needs, to do so open vocalmat_identfier, under the vocalmat_identifier subfolder of the master, navigate to ~line 140. Replace F>30000 with whatever you find suitable, for instance if you want VocalMat to ignore detections below 25kHz you would write F>25000
- Recording protocol: follow the protocol established by Ferhat et al, 2016.
- Sampling rate: we recommend using a sampling rate of 250kHz (Fmax=125kHz).
- MATLAB: versions 2017a through 2019b. For other versions refer to the FAQ.
- MATLAB Add-Ons:
- Signal Processing Toolbox
- Deep Learning Toolbox
- Image Processing Toolbox
- Statistics and Machine Learning Toolbox
- Will
VocalMatwork with my MATLAB version?
VocalMat was developed and tested using MATLAB 2017a through 2019b versions. We cannot guarantee that it will work in other versions of MATLAB. If your MATLAB version supports all the required Add-Ons, VocalMat should work. It should be noted that VM4Rats was adapted and trained using MATLAB 2020b - 2021b on Windows 10.
- What are the hardware requirements to run
VocalMat?
The duration of the audio files that can be processed in VocalMat is limited to the amount of RAM your computer has. We estimate around 1GB of RAM for every minute of recording using one minute segments. For a 10 minute recording, your computer should have at least 10GB of RAM available. RAM usage will vary depending on your MATLAB version and computer, these numbers are just estimates.
- Will
VocalMatwork with my HPC Cluster?
In order for our script to work in your Cluster it must have Slurm support. Minor changes might have to be made to adapt the script to your Cluster configuration.
- I want a new feature for
VocalMat, can I contribute?
Yes! If you like VocalMat and want to help us add new features, please create a pull request!