Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Music Exploration



Python 3.7+


Copy the config file example to instance folder

mkdir instance
cp instance/

Edit the config:

  • Point the ROOT_DIR to the empty directory where all the data will be stored
  • Point the AUDIO_DIR to the directory with all the audio files

If you are using own collection of audio, to serve audio from local server create a soft link app/static/audio pointing to your audio folder and make sure that AUDIO_PROVIDER is set to local.

ln -s /path/to/your/audio app/static/audio

If you want to visualize your local collection, you can run that will basically execute all the next steps automatically.

If you are using mtg-jamendo-dataset, you can serve audio directly from Jamendo servers by registering an app in Jamendo Dev portal and setting JAMENDO_CLIENT_ID variable. In this case make sure that AUDIO_PROVIDER is set to jamendo.


python3.x -m venv venv
source venv/bin/activate
pip install --upgrade pip setuptools wheel
pip install -r requirements.txt

If you get an error while installing annoy, make sure that you have python3.x-dev installed

You can use python 3.8+ with no problems for running the app, but you will need separate environment for audio processing, as, there are no essentia-tensorflow wheels for Python 3.8 yet. However, it is a good idea to have separate environments for processing and running anyway, as there are some packages that are only used for processing.

Additional processing libraries:

  • tensorflow-essentia: extracting embeddings (has no wheels for Python 3.8 yet, 380MB):
pip install essentia-tensorflow
  • tinytag: install it if you use personal music collection, it parses ID3 tags
pip install essentia-tensorflow tinytag

Download essentia-tensorflow models

mkdir essentia-tf-models
cd essentia-tf-models
wget -O msd-musicnn.pb
wget -O msd-vgg.pb
wget -O mtt-musicnn.pb
wget -O mtt-vgg.pb
wget -O audioset-vggish.pb

If you don't want to use all models, feel free to comment out entries in app/models.yaml

Embedding layers for the models:

Model Layer Size
MusiCNN model/batch_normalization_10/batchnorm/add_1 200
VGG model/flatten/Reshape 2 x 128 = 256
VGGish model/vggish/fc2/BiasAdd 128

If you don't have a lot of tracks, feel free to add tsne to offline_projections in app/models.yaml. By default t-SNE is applied dynamically only on the number of tracks that you are visualizing at a time.

Process audio

To take full advantage of this application, make sure your audio is in the format that your browser can reproduce (mp3, ogg, etc.). If you need to batch convert your audio, check out this gist

flask init-db  # creates tables in db
flask index-all-audio  # creates list of audio tracks in db
flask extract-all essentia-tf-models  # extracts embeddings
flask reduce-all  # computes the projections
flask index-all-embeddings  # indexes everything in database
flask aggregate-all # aggregates embeddings in single .npy file per model (to get rid of many small files)

Adding local metadata

flask load-id3-metadata

Adding Jamendo metadata

Clone or download metadata from mtg-jamendo-dataset

flask load-jamendo-metadata path/to/mtg-jamendo-dataset/data/raw_30s_cleantags.tsv
flask query-jamendo-metadata

Creating playlists

If you are using nix-based system, the playlist creation should work out of the box. If you want to create playlists for later use, change PLAYLIST_FOR_OFFLINE=True in

If you are using Windows Subsystem for Linux (WSL) and would like to use playlists in Windows, apart from setting PLAYLIST_FOR_OFFLINE=True, you should also set PLAYLIST_USE_WINDOWS_PATH = True and the path PLAYLIST_AUDIO_DIR = 'C:\\path\\to\\audio' that is the same one that you created symbolic link before, but as Windows path.

Note: none of these options have any effect if the AUDIO_PROVIDER='jamendo'.

Running the app

FLASK_ENV=development flask run

Deploying with Docker (to be updated)

  • Set SERVE_AUDIO_LOCALLY in appropriately depending on if you want to use Jamendo API or not
  • Build and run docker image with your data mounted at /data, it uses port 80 by default
  • If SERVE_AUDIO_LOCALLY=True, make sure to mount your audio dir at /app/static/audio otherwise make sure that you set JAMENDO_CLIENT_ID environment variable


docker build -t music-explore .
docker run -p 8080:80 -v /path/to/data:/data -v /path/to/audio:/app/static/audio music-explore  # run with local audio
docker run -p 8080:80 -v /path/to/data:/data --env JAMENDO_CLIENT_ID=XXXXXXXX music-explore  # run with Jamendo API


pip install pre-commit
pre-commit install


The code is licensed under GNU Affero General Public License v3.0.


When using or referencing this work, please cite the following publication:

  title = {Web Interface for Exploration of Latent and Tag Spaces in Music Auto-Tagging},
  author = {Philip Tovstogan and Xavier Serra and Dmitry Bogdanov},
  booktitle = {Machine Learning for Media Discovery Workshop (ML4MD), International Conference on Machine Learning (ICML)},
  year = {2020}


This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068.

Flag of Europe


App to explore latent spaces of music collections







No packages published