audio-segment

MFCC based audio grain sorting

How this script works:

1. Global parameters:

• number of MFCCs used,
• size of the FFT used in calculating MFCCs,
• length of the frame,
• 0th MFCC toggle,
• onset detection toggle,
• windowing toggle,
• smoothing toggle,
• smoothing window width,
• sigma parameter of the Gauss filter,
• segmentation hop length.

2. Loading a file using GUI. [ipyfilechooser] / [google.colab]
3. Segmentation:

• into frames of set width, [librosa]
• on onset times. [librosa]

4. Windowing of signal in each frame using Hamming Function. [numpy]
5. Calculation of MFCCs for each frame without centering [librosa]
6. Example frame sorting using MFCC values via as folllows,

• using a k-d tree structure: [scipy.spatial]
  • by querying for all the neighbours of frame[0], sorted by distance in ascending order,
  • by finding the shortest path that traverses the entire tree starting from frame[0],
• using correlation clustering represented as hierarchy dendrogram, sorted by distance in ascending order. [scipy.cluster]

7. Output construction via concatenation of frames in arrangements calculated in step 6., optionally smoothing out signal discontinuities between them using a 1-Dimensional Gaussian filter. [scipy.ndimage]

All results displayed are plotted using matplotlib.

This project references online resources which elaborate further on the concepts of audio feature extraction and math-based workflows in Python, such as musicinformationretrieval.com as well as documentation of matplotlib, numpy, librosa and scipy libraries.