The official repository for the paper "NeuProNet: Neural Profiling Networks for Sound Classification".
Real-world sound signals exhibit various aspects of grouping and profiling behaviors, such as being recorded from identical sources, having similar environmental settings, or encountering related background noises. In this work, we propose novel neural profiling networks (NeuProNet) capable of learning and extracting high-level unique profile representations from sounds. An end-to-end framework is developed so that any backbone architectures can be plugged in and trained, achieving better performance in any downstream sound classification tasks. We introduce an in-batch profile grouping mechanism based on profile awareness and attention pooling to produce reliable and robust features with contrastive learning. Furthermore, extensive experiments are conducted on multiple benchmark datasets and tasks to show that neural computing models under the guidance of our framework gain significant performance gaps across all evaluation tasks. Particularly, the integration of NeuProNet surpasses recent state-of-the-art (SoTA) approaches on UrbanSound8K and VocalSound datasets with statistically significant improvements in benchmarking metrics, up to 5.92% in accuracy compared to the previous SoTA method and up to 20.19% compared to baselines. Our work provides a strong foundation for utilizing neural profiling for machine learning tasks.
This implementation is based on the fairseq toolkit. Our main source code was written in the following directory:
- Spectrum Data Loader
- NeuProNet framework and backbones implementation
- NeuProNet loss function
- Configuration files for training baselines with or without NeuProNet
Please follow the instructions to install the framework.
Additionally, install librosa and efficientnet_pytorch:
pip install librosa soundfile efficientnet_pytorch
Without NeuProNet:
fairseq-hydra-train task.data=$DATA_PATH task.profiling=False --config-dir examples/NeuProNet/config/finetuning --config-name urbansound8k
fairseq-hydra-train task.data=$DATA_PATH task.profiling=False --config-dir examples/NeuProNet/config/finetuning --config-name vocalsound
With NeuProNet:
fairseq-hydra-train task.data=$DATA_PATH task.profiling=True --config-dir examples/NeuProNet/config/finetuning --config-name urbansound8k
fairseq-hydra-train task.data=$DATA_PATH task.profiling=True --config-dir examples/NeuProNet/config/finetuning --config-name vocalsound
For evaluating models trained on VocalSound:
fairseq-validate --path $TRAINED_MODEL_PATH --task audio_finetuning $DATA_PATH --valid-subset test --batch-size 128
For evaluating models trained on UrbanSound8k:
fairseq-validate --path $TRAINED_MODEL_PATH --task audio_finetuning $DATA_PATH --valid-subset valid --batch-size 512