This project is an environment for conducting analysis with the aim of uncovering the determining factors and patterns in the composition of house music. The related research paper for this project can be found here.
.
├── Dockerfile # For Omnizart
├── docker-compose.yml # For Omnizart
├── requirements.txt # Required libraries for Python programs
├── NATTEN/ # Models required for All-In-One usage
├── data/ # Experimental data
├── images/ # Experimental results
├── notebook/ # Jupyter notebook environment for experimental programs
└── src/ # Experimental programs
└── scripts/ # Various scripts
data/ contains only a partial public dataset due to copyright restrictions.
To run the programs in src/, you need to install the required libraries:
Alternatively, programs with the same content as src/ can be found in notebook/experiments.ipynb.
You need to place experimental data in the data/ directory.
In this project, the composition of a house music track is defined into the following four sections:
| Section | Description |
|---|---|
| Intro | Introduction part of the track |
| Drop | Climax part of the track |
| Break | Part other than the climax |
| Outro | Conclusion part of the track |
Analysis of the relationship between each section and frequency feature (Spectral Centroid).
- Retrieve segmentation information of music using All-In-One.
- Extract frequency feature (Spectral Centroid) from the music.
- Visualize the average frequency feature values for each section based on segmentation information.
Analysis of the relationship between each section and frequency feature (Spectral Centroid) for each part of the music.
- Retrieve segmentation information of music using All-In-One.
- Perform source separation of the music into 4 parts (drums, bass, other, vocals) using Demucs.
- Exclude source intervals where RMS falls below a threshold.
- Extract frequency feature (Spectral Centroid) from the music.
- Visualize the average frequency feature values for each section and each part based on segmentation information.
Investigation of effective durations for each part in experiment2.
- Retrieve segmentation information of music using All-In-One.
- Perform source separation of the music into 4 parts (drums, bass, other, vocals) using Demucs.
- Calculate the length (effective duration) of source intervals where RMS exceeds a threshold.
- Visualize the effective duration for each section and each part.
Analysis of the relationship between each section and sound pressure feature (RMS) for each part of the music.
- Retrieve segmentation information of music using All-In-One.
- Perform source separation of the music into 4 parts (drums, bass, other, vocals) using Demucs.
- Extract sound pressure feature (RMS) from the music.
- Visualize the average sound pressure feature values for each section and each part based on segmentation information.
Analysis of the sound pressure balance trends for each part in each section.
- Retrieve segmentation information of music using All-In-One.
- Perform source separation of the music into 3 parts (drums, bass, other+vocals) using Demucs.
- Extract sound pressure feature (RMS) from the music.
- Plot the sound pressure feature balance for each section in a 3D space.
Analysis of the trends in drum elements for each section.
- Use Omnizart to extract drum MIDI from audio sources.
- Aggregate the number of various drum elements for each section and visualize them.
Analysis of the trends in the timing of drum element activations for each section.
- Use Omnizart to extract drum MIDI from audio sources.
- Aggregate the number and timing of various drum elements for each section.
- Visualize the timing of drum element activations as spaghetti plots.
Analysis of the trends in the timing of drum element activations for each section.
- Use Omnizart to extract drum MIDI from audio sources.
- Calculate the length of 8 bars of the music using All-In-One's tempo estimation results.
- Aggregate the number of various drum elements for each section in 8-bar segments.
- Visualize the number of drum elements in 8-bar segments as spaghetti plots.
Analysis of the correlation between drum pattern changes and section changes.
- Use Omnizart to extract drum MIDI from audio sources.
- Retrieve section change timings from All-In-One's segmentation information.
- Detect drum pattern changes by measuring the lengths between all drum events in MIDI and using the standard deviation as a threshold.
- Calculate the coincidence rate between drum pattern change timings and section change timings for cases where each is the denominator.
- Visualize the coincidence rate.
Probability with drum pattern changes as the denominator:
Probability with section changes as the denominator:
Justin Wulf - wulf@kthrlab.jp