Automated melodic dictation
Matlab M
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
doc
images
sound
.gitignore
H_to_midi.m
README.md
automatic_H_to_midi.m
draw_midi.m
f_mesure.m
find_note_endings.m
makeW.m
melodic_dictation.m
nmf.m
plot_image.m
tfct.m

README.md

melodic-dictation

This project is a matlab script for automated melodic dictation.

How it works

First, the spectrogram of the music track is extracted using a Short-Term Fourier Transform (STFT, or TFCT).

STFT

Then, the idea is to break down the spectrogram matrix V as a product of two matrices: V = W*H Where:

  • W is a preconstituted matrix containing the spectrograms of all the notes in the chromatic scale
  • H is a matrix describing which notes are played at all times - a.k.a the music score.

chromatic scale

So while we can easily generate W, the goal is to find H using the Non-negative Matrix Factorization (NMF) method. An algorithm using the Kullback-Leibler divergence is described in a paper by Lee and Seung.

Applied to the previous spectrogram, we get the following H matrix:

Calculated H

Some signal processing will help to determine the start and end of each note:

Filtered H

And we can finally export the result to a music score or a MIDI file

Music score

A few examples

Original Reconstruction
tetris.mid tetris_out.mid
zelda.mid zelda_out.mid
mario.mid mario_out.mid