This is an accompanying GitHub page for research on computationally characterizing and visualizing drum machine samples.
Requirments: Python 3, Essentia, MySQL
MySQL: https://dev.mysql.com/doc/refman/8.0/en/installing.html
Essentia: http://essentia.upf.edu/documentation/installing.html
Install python denpendencies
pip install numpy matplotlib sklearn pandas
pip install django mysqlclient
Ceate a new MySQL database with the correct permissions for Django
mysql < ./sample_analysis/db_utils/setup_db.sql
All commands are setup through the django command interface. They can be accessed using:
cd sample_analysis
python manage.py command arguments [optional-arguments]
For more information on Django commands: https://simpleisbetterthancomplex.com/tutorial/2018/08/27/how-to-create-custom-django-management-commands.html
In order to create tables in the database for each model, run:
python manage.py migrate
Project should now be ready to start loading in samples!
python manage.py loadsamples directory
SAMPLES MUST BE ORGANIZED CORRECTLY
A particular folder structure is assumed so that samples are correctly organized by Sample Pack (Drum Machine), Kit, and Sample Type (kick or snare)
This structure is as follows:
samples/drum_machine/kit_x/kick/sample_name.wav
or
samples/drum_machine/kick/sample_name.wav
if there are not multiple kits.
Replace kick with snare for snare drum samples.
python manage.py feature_extraction [--window_length] [--window_start]
--window_length and --window_start are optional arguments that can be used to specify
the time segmentation scheme that is applied to each audio sample prior to audio feature extraction.
Window length is given in milliseconds and window start is given in the percantage of the attack
portion of the audio signal.
If both window length and window start are left out, then the full sample length will be used starting from the beginning of the sample. This will be stored in the database as a window length and window start of 0 and 0.
Audio feature extraction is performed using Essentia and 133 features are extracted for each audio sample.
Alternatively, there is a shell script can be executed that will run a full feature extraction on a selection of window lengths and start times that were used for this research.
./scripts/feature_extraction.sh
Results of audio feature extraction are stored in the data model Feature
Principal Component Analysis (PCA) is used here as an analysis tool to evaluate the effects of time segmentation prior to audio feature extraction.
To run PCA on a set of samples:
python manage.py pca sample_type window_length window_start
Where sample_type is either 'ki' or 'sn', window_length is the window length in ms, and window_start is the starting time of the window in percentage of the attack. The combination of window_length and window_start must have already been extracted during feature analysis in the previous step.
Tests for sample adequecy and the null hypothesis are performed and the outputs for these are printed to stdout. Tests performed are Bartlett's Test of Sphericity, Levene's Test, and Kaiser-Meyer-Olkin.
Explained variance ratio, explained variance, and top components are printed to stdout. The first four dimensions resulting from PCA for each sample are saved as a PCA object in the database and the explained variance ratio is saved as a PCAStat object in the database. Each PCAStat data object has a unique sample type, window length and window start.
There are two shell scripts that will run PCA on all window length and start variations used in the research. They can be executed using:
./scripts/pca_kick.sh && ./scripts/pca_snare.sh
python manage.py pca_mixed sample_type
In this scheme, multiple combinations of window_length and window_start are used in order to maximize the variance for each dimension across all samples of a particular type. SamplePCA objects generated from this command will be stored with a window_start and and window_length of -1.
Manifold learning is used here for dimension reduction to 2 dimensions for visual plotting. The command to perform manifold learning is:
python manage.py manifold manifold_method sample_type window_length window_start
For mixed time segmentations:
python manage.py manifold_mixed manifold_method sample_type
Where manifold_method is one of tsne, tsne_pca, isomap, locally_linear, mds, or spectral
For more information on each of these manifold learning methods see: https://scikit-learn.org/stable/modules/classes.html#module-sklearn.manifold
Results of manifold learning dimension reduction are stored as a Manifold object in the database.
To run all manifold methods on all combinations of window start and length used in this research use the shell scripts:
./scripts/manifold_ki.sh && ./scripts/manifold_sn.sh
Classification is used to evaluate the windowing methods as well as to evaluate the effects of the various dimension reduction techniques. Classification is performed using SVC, Perceptron, and Random Forest methods. For each classification task, the baseline score is printed, along with the score achieved by each individual method, and then an average of the three methods. Each of these scores is saved to the database as a Classification object.
Classification between kicks and snares using the full dimension dataset
python manage.py classifier_sample_type window_length window_start
Mixed Time Segmentation:
python manage.py classifer_sample_type_mixed
To run classification on all time segmentations used in this research:
./scripts/sample_type_classification.sh
Runs classification on drum machine type within sample type (ie. only on kicks, or only on snares). Only uses drum machines that contain over 50 samples.
python manage.py classifier_dm sample_type window_length window_start
Mixed time segmentation:
python manage.py classifier_dm_mixed sample_type
To run drum machine classification on all time segmentations used in this research:
./scripts/drum_machine_classification_ki.sh && ./scripts/drum_machine_classification_sn.sh
Drum Classification on two dimensions after manifold learning This runs the same drum machine classification tests, but using only two dimensions resulting from a particular manifold learning technique:
python manage.py classifier_dm_reduced reduction_method sample_type window_length window_start
Where reduction method is one of the manifold methods or pca
To run drum machine classification on all time segmentations and manifold learning methods:
./scripts/drum_machine_classification_reduced_ki.sh && ./scripts/drum_machine_classification_reduced_sn.sh
Runs classification on a set of kick or snare samples produced by the same manufacturer.
To link drum machines with manufacturers in the database, the follow command needs to be run first:
python manage.py add_manufacturers
This will add the following manufacturers: Alesis, Boss, E-Mu, Korg, Roland, and Yamaha. Drum machines with these names will be automatically linked to these manufacturers.
To run manufacturer classification on these drum machines:
python manage.py classifier_manu sample_type window_length window_start
With mixed time segmentations
python manage.py classifier_manu_mixed sample_type
To run on all time segmentation choices:
./scripts/manufacturer_classification_ki.sh && ./scripts/manufacturer_classification_sn.sh
python manage.py plot reduction_method sample_type window_length window_start
This will use matplotlib to plot the samples used in the drum machine classification task.