Fake Music

The aim is to let the computer generate nice music for us.

Roadmap

• Information Retrieval (Tools, ...)
• Converting Input Data
  • Parse single files
  • Parse all files of a folder
• Preprocessing
  • Normalization, One-Hot Encoding, ...
  • Save label encoding
  • Prepare input/output data
  • Reshape input data
• Neural Network
  • Create Layers
  • Fit Model
  • Compile Model
  • Predict Model
  • Plot Model
• Building a LSTM
  • Layer Architecture
  • Callbacks (Keras ModelCheckpoints)
  • Configuration
  • Prediction
• Generate Music Data
  • Model prediction
  • Decoding predicted results
  • Convert back to Music File Format (MIDI, MP3, ...)
• Optimization
  • Use only a single instrument
  • Add chords
  • Add duration
  • Add offset
  • LSTM optimization
  • Converting midi to mp3 and export
  • JSON file to load all the settings from

Getting Started

First of all, make sure you have all the dependencies installed (see Setup).
Once you completed this step, you should be able to run the main.py file with Python 3.

There are default folder structures set. You can see them running:

python3 main.py -h

The help section also shows you possible configurations.
To change batch size or similar attributes, you currently have to edit the config.py file accordingly.

There is an easy way to test this software!
We implemented a web application that provides a simple interface and allows you to test this software
without having to write lots of console commands.
To use it, just make sure that the main.py of the server can be executed
by installing possible missing dependencies.
You can find a README in the server folder.
Once started, the interface will be accessible at:

localhost:8080/

Setup

One requirement is that you have Python 3 installed.
This code is based on it.

You can install each python dependency manually or just run...

pip install -r requirements.txt

...in the according folder (for example in ./midi-parser).

The manual installation progress is described below.

Manual Installation

Installing Keras

pip install keras

Installing Music21

pip install music21

Installing Scikit

pip install scikit-learn
pip install sklearn

Installing Flask

pip install flask
pip install flask-socketio

Installing Tensorflow-GPU (v 1.8.0)

pip install tensorflow-gpu

Installing Matplotlib

pip install matplotlib

If you just want to run the software without GPU-support, just install tensorflow.
This dependency may already be installed by keras.

If you have trouble executing the software (libcublas error or similar), make sure you have the NVIDIA CUDA TOOLKIT
as well as the correct NVIDIA CUDNN version installed on your system.

If things still dont work, have a look at the error section.

Features

Note

For training, we use the following information:

• Duration
• Pitch
• Offset

Web-Service

We currently work on integrating a web service that will provide a simple interface.
This interface allows to upload a single or multiple midi files and change network settings.
After the configuration, the user will have the option to submit this data to the server.
It will then start a training process in a new thread and show the results.

The files as well as the README are located in the server folder.

Useful Links

• TensorFlow - RNNs
• TensorFlow GPU Installation
• Keras
• Music21 - DOC
• Generating Music with an LSTM
• Table of Pitch Notations
• One-Hot Encoding Explained
• Introduction to Cross-Entropy Loss
• RMSProp Optimizer
• Neural Networks FAQs
• Deep Learning Glossary (Iterations vs Epochs...)
• MIDI File Format and MIDI File Format Specification and MIDI Format
• Bidirectional Usage

More Useful Links

• FREEMIDI - TOP
• MIDI - Visualizer 1
• MIDI - Visualizer 2
• C++ Midifile Parser - GIT
• Run C++ Program and get its output (Python) - DOCS
• MIDI-JSON-API
• Understanding LSTM Networks
• Keras NP_UTILS
• MIDI File Time Division
• MIDI Event List
• Attention Mechanism
• Bidirectional Wrapper
• Optimizer Comparison

Known Problems

Fix Cuda Problem (Linux)

Type in terminal before executing the program

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

If you have trouble finding the correct versions, see here.
For example: tensorflow-gpu-1.4.1 uses CUDA-8.0 and CUDNN-6.0

API Documentation

• Scikit - Preprocessing
• Keras - Layers
• Pandas - Indexing and Selecting Data

MIDI Specifications

Graphic Source: http://www.cs.uccs.edu/~cs525/midi/midiFileFormat.png