Estimation of size and age of Eurasian Snowfinch broods based on audio recordings
Clone the repository and then initialize the submodules with the command
git submodule update --init --recursive
Create Python 3.9 venv and install the dependencies with the commands
python3.9 -m venv venv
. ./venv/bin/activate
pip install -r begging-analyzer/requirements.txt
pip install -r requirements.txt --force-reinstall
Run the script install.ps1 with PowerShell in order to install all the dependencies.
In order to run predictions, use classify.py
script placed in the project root directory. The script accepts a
few parameters, which can be supplied by specifying a path to a configuration file via -c
command line option.
If the configuration file path is given by -c
option, there is no need for other command line arguments, as long
as the supplied configuration provides all required parameters. The schema of configuration file is described in the
following subsection.
If -c
option is ommitted, the following command line options are required:
-i <input_path>
- a path to input data; it can be an audio file, a directory with audio files or a directory with feeding statistics (in case of a simple brood size classifier),-m <model_path>
- a path to a serialized trained model or a directory containing such model files; if directory with multiple model files is specified, the program will perform classification using each of the models and store results in corresponding subdirectories.
There are also several optional parameters:
-l <label_path>
- a path to a directory with label files, structured in the same way as the recordings ininput_path
; this option is not required, if not specified, the program will assume that label files are placed next to the corresponding recordings,-o <output_directory>
- a path to a directory where prediction results are to be stored,_out
by default,-w <n_workers>
- number of parallel processes to use for classification, 10 by default,-p <period>
- a period (in hours) of results aggregation, 48 by default,--overlap-hours <overlap>
- an overlap for aggregation period given in hours, 0 by default,--mt-threshold <threshold>
- a threshold used in multi target classification to determine whether a prediction score indicates belonging to a class or not; used only when a multi target capable model is supplied.
The program configuration can be specified in a form of a YAML file with the following schema:
data:
recordings: <path>
labels: <path>
model:
path: <path>
n_workers: <integer>
output_dir: <path>
aggregation:
period_hours: <integer>
overlap_hours: <integer>
feeding_detector:
type: <type>
path: <path>
args:
model_path: <path>
extension: <extension>
...
The following attributes are required:
data.recordings
- same as-i
command line option,model.path
- same as-m
command line option.
All the other attributes are optional and each of them corresponds to one of the command line options described above.
The input directory for a simple brood size classifier should contain the following files:
- feeding-stats.csv - a file with feeding statistics produced by SnowfinchWire.BeggingCallsAnalyzer,
- brood-age.csv - a file with age information; this file is produced by running age classification with
a CNN model - it is called brood-period-preds.csv; the following columns are required:
- brood_id - brood identifier, string
- period_start - date and time in ISO format
- class - age of the youngest nestling, floating point number
The program is capable of generating feeding labels for recordings by calling SnowfinchWire.BeggingCallsAnalyzer interface. If the user wishes to do this, they can use the following options:
--feeding-detector-model <path>
- a path to a serialized trained feeding detector model,--feeding-detector-path <path>
- a path to standalone SnowfinchWire.BeggingCallsAnalyzer installation, if not specified embedded installation will be used,--feeding-detector-type <type>
- a type of feeding detector to use; available options arefe
(feeding extraction) andoss
(OpenSoundscape),fe
by default.
Alternatively, feeding detection can be configured via the configuration file described above, using the
feeding_detector
object. The type
attribute is equivalent to --feeding-detector-type
command line option and
the path
attribute is equivalent to --feeding-detector-path
option. All attributes defined under args
key are
transparently passed to the SnowfinchWire.BeggingCallsAnalyzer launcher.
It is possible to perform classification with OpenSoundScape model for age detection (age-oss model) nad simple brood size classifier (size-stat model) in a single run. In order to do this, both models should be placed under one directory passed to the program as model_path. For the program to work correctly, it is required that the size-stat model file name contains "size-stat" phrase and the age-oss model file name contains "age-oss" phrase. The program will first run feeding detection (if specified), then brood age classification and finally brood size classification, since this process uses output of both the previous as its input.
There are a few trained models available on SharePoint under Documents/BroodAnalyzerModels directory:
- 20240402-Age
- age-oss-model-inception.zip
- 20240422-Size
- size-stat-model-n20-vs-svm-bayes.zip
- size-stat-model-n20-vs-svm-bayes-rfc.zip
The models for size classification have slightly different accuracy scores on test data set for different class selection:
- size-stat-model-n20-vs-svm-bayes.zip
- classes 2, 3, 4, 5: 0.569,
- classes 2-3, 4-5: 0.875,
- classes 2-3, 4, 5: 0.815,
- classes 2, 3-4, 5: 0.666,
- classes 2, 3, 4-5: 0.636
- size-stat-model-n20-vs-svm-bayes-rfc.zip
- classes 2, 3, 4, 5: 0.522,
- classes 2-3, 4-5: 0.881,
- classes 2-3, 4, 5: 0.804,
- classes 2, 3-4, 5: 0.668,
- classes 2, 3, 4-5: 0.609
In order to train a model, use train_model.py
script located under the project root directory. The script requires
several command line arguments and options, which partly depend on the model type. The common argument is
data_path
- a path to a directory containing the dataset definitions. There are also a few common options:
-
-c <path>
- a path to dataset split configuration file; all such files used for experiments conducted so far are located underconfig
directory; this is a required option, -
-a <architecture>
- a model architecture to use, can be one of the following:- simple-ensemble to train simple brood size classifier,
- One of CNN architectures supported by OpenSoundscape:
- resnet18,
- resnet50,
- resnet101,
- resnet152,
- vgg11_bn,
- densenet121,
- inception_v3,
- matchboxnet.
Set to resnet18 by default,
-
--out <output_directory>
- a path to a directory where prediction results are to be stored,_out
by default.
In order to train any model, brood information file is necessary. It is a .csv file with the following columns:
- brood_id - brood identifier, string
- datetime - date and time in ISO format
- age_min - age of the youngest nestling, floating point number
- age_max - age of the eldest nestling, floating point number
- size - true brood size, integer
A single line of the file should describe one of the broods used in the training process at a specific time. For each brood there should be at least one line per day of recordings.
If the user chooses to train the simple size classifier, they can specify the following additional options:
--n-simple-models <count>
- a count of simple models of each type used in a target ensemble classifier, 20 by default,--ensemble-voting <type>
- a type of voting used by an ensemble model to determine final predictions; can be set to soft (default) or hard.--ensemble-svm
- use SVM models to compose ensemble--ensemble-rfc
- use Random Forest models to compose ensemble--ensemble-mlp
- use MLP models to compose ensemble--ensemble-bayes
- use Bayesian classifiers to compose ensemble
Note: At least one of flags
--ensemble-svm
,--ensemble-rfc
,--ensemble-mlp
,--ensemble-bayes
has to be supplied.
The dataset definition directory has to contain the following files:
- feeding-stats.csv - a file with feeding statistics produced by SnowfinchWire.BeggingCallsAnalyzer,
- snowfinch-broods.csv - a brood information file described in the previous section
Training a CNN model from OpenSoundscape framework requires an additional command line option: --audio-path <path>
.
It has to specify a path to a directory containg audio recordings. Moreover, there are a few more options:
-d <duration>
- an audio sample duration in seconds for the model to work on, 2 by default,-n <n_epochs>
- a number of training epochs, 10 by default,-w <n_workers>
- a number of parallel processes used for training,-b <batch_size>
- a batch size, 100 by default,-l <learning_rate>
- a learning rate, 0.001 by default,-t <target>
- classification target: size, age or all (default); if all is chosen, two models are generated,--age-mode <mode>
- classification mode for brood age: single (default) or multi; use this option to train multi target model,--samples-per-class <n>
- specifies how many audio samples should be used per each class; by default the quantity of samples representing the least numerous class is used.
The dataset definition directory has to contain the following files:
- brood-size.csv
- brood-age.csv
These files can be created using the following commands:
python -m sfw_brood.describe_recordings \
--brood-data <path/to/brood-information-file.csv> \
-o <path/to/snowfinch-recordings.csv> \
<path/to/recordings>
python -m sfw_brood.export_data \
-s <sample duration in seconds> \
-o <sample overlap in seconds> \
-j <number of processes> \
--rec-data <path/to/snowfinch-recordings.csv> \
--audio-out-path <path/to/audio-output> \
<path/to/recordings> <path/to/output>
The first command will produce the file snowfinch-recordings.csv in a location specified by the user and the
second will use it to prepare data for OpenSoundscapre training process. As a result, the recordings will be sliced
to samples of the given length with the requested overlap. The samples will be stored under the path specified by
the --audio-out-path
option. The brood-size.csv and brood-age.csv files will be created under the path
specified by the last argument.