JavaScript Python Other
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.

Neural music pad

A variational autoencoder (VAE) is used to generate outputs (X) from a continuous latent space (Z) by learning a generative model p(X | Z). In our case we want to generate music by controlling the position of the user's cursor. So we let X(t) be an 88-d binary vector that specifies which of 88 different notes to play at some time t, and Z(t) is controlled by your mouse.

How it works

This variational autoencoder was trained on 382 of Bach's four-part chorales (source), transposed to Cmaj or Amin. The model was built and trained using keras. Code for this process can be found here.

In the browser, I'm using p5.js to play sound and handle mouse clicks. In the backend, I use the model loaded in keras to generate notes from the position clicked on the pad. I then use music21 to detect which chord those notes correspond to.

Running locally

First, install requirements with pip install -r requirements.txt. Then run python and navigate in your browser to