MINGUS - Melodic Improvisation Neural Generator Using Seq2seq

Pretrained weights (jazz)

Weights for the WJazzD-trained MINGUS (used by the comparison pipeline) live on Hugging Face: maxkudryashov/mingus-1.

Two ablation runs (different conditioning), both 10 epochs / BPTT 35 / split 340/42/40:

run	pitch cond	duration cond	pitch test_ppl	pitch test_acc	duration test_ppl	duration test_acc
paper/ (default in train.py)	I-C-NC-B-BE-O	I-C-NC-B-BE-O	13.08	0.131	4.087	0.344
paper-optimal/ (per ISMIR 2021 §3.2)	D-C-B-BE-O	B-BE-O	13.53	0.120	4.048	0.345
Madaghiele 2021 (paper, on 86-solo split)	D-C-B-BE-O	B-BE-O	11.01	0.163	4.140	0.323

The paper-optimal swap didn't help our pitch numbers — most likely because our test set is 40 specific solos (cross-model intersection) rather than the paper's 86 random ones, making the conditioning-search space test-set-specific. Duration model reproduces / slightly beats paper in both runs, confirming the pipeline is correct.

Download (paper-optimal — recommended):

pip install -U huggingface_hub
hf download maxkudryashov/mingus-1 \
  "paper-optimal/pitchModel/MINGUS COND D-C-B-BE-O Epochs 10.pt" \
  "paper-optimal/durationModel/MINGUS COND B-BE-O Epochs 10.pt" \
  --local-dir B_train/models
# → B_train/models/paper-optimal/{pitchModel,durationModel}/MINGUS COND ... Epochs 10.pt

Per-epoch metrics, summary.json, and train.log:

• results/mingus-wjazzd-10ep/ — default conditioning
• results/mingus-wjazzd-paper-optimal-10ep/ — paper-optimal conditioning

Trained via the Colab notebook at training/colab_train.ipynb.

---

MINGUS is a Transformer-based Seq2Seq architecture for modelling and generating monophonic jazz melodic lines. MINGUS relies on two dedicated embedding models (respectively for pitch and duration) and exploits in prediction features such as chords (current and following), bass line, position inside the measure. The obtained results are comparable with the state of the art of music generation with neural models, with particularly good performances on jazz music.

More information in the paper (bib):

Vincenzo Madaghiele, Pasquale Lisena, Raphaël Troncy. MINGUS: Melodic Improvisation Neural Generator Using Seq2Seq. In Proceedings of 22nd International Society of Music Information Retrieval Conference (ISMIR), 8-12 November 2021, Online.

The web application for user evaluation is available at mingus.tools.eurecom.fr.
Supplementary material with additional results about the paper is available here.
You can listen to jazz melodies generated by MINGUS here
An interactive browser demo of the midi generation is available here

Abstract:

0. Install dependencies

To use the model run the following code from the main project directory. After downloading this repository, run the following code in the terminal to install the required libraries in a new conda environment named MINGUS and activate it:

$ conda env create -f environment.yml
$ conda activate MINGUS

If this installation does not work correctly you can install the dependencies listed in the file requirement.txt individually. It will probably be necessary to run the following command:

$ export PYTHONPATH="$PWD"

1. Pre-process data

To train MINGUS with a custom dataset place your musicXML data in 00_preprocessData/data/xml and run the following command in your terminal:

$ python3 A_preprocessData/data_preprocessing.py --format xml

This will generate the file 00_preprocessData/data/DATA.json which will be used for training. This file should be the same for all the following commands (even generation and evaluation). WjazzDB requires specific pre-processing from the csv files provided by its authors. To train with WjazzDB run:

$ python3 A_preprocessData/wjazzDB_csv_to_xml.py
$ python3 A_preprocessData/data_preprocessing.py --format xml

2. Train MINGUS

To train MINGUS from scratch run:

$ python3 B_train/train.py

The saved model will be in B_train/models. It is possible to visualize training details with tensorboard by running:

$ tensorboard --logdir=B_train/runs

The results are showed in your browser at http://localhost:6006/.

3. Generate music

To generate music with MINGUS you can run the following command. Please ensure that the arguments correspond to an already trained model. You can use the files in C_generate/xml4gen as a starting point. These files are also used for evaluation.

$ python3 C_generate/generate.py --xmlSTANDARD <path_to_xml_standard>

Output midi files will be in C_generate/generated_tunes.

To generate with our pre-trained model run:

$ python3 C_generate/generate.py --EPOCHS 100 --xmlSTANDARD <path_to_xml_standard>

4. Evaluate results

The evaluation metrics are computed by comparing the melodies generated by MINGUS and a corpus of original samples, so to evaluate the model you should first choose some reference xml files and put them in D_eval/reference (you can randomly select some files from your original dataset). Then you should generate a corpus of melodies with MINGUS and evaluate the results:

$ python3 C_generate/generate.py --GENERATE_CORPUS
$ python3 D_evaluate/evaluate.py

The evaluation metrics are available in D_evaluate/metrics.

To evaluate our pre-trained model run:

$ python3 C_generate/generate.py --GENERATE_CORPUS --EPOCHS 100
$ python3 D_evaluate/evaluate.py --EPOCHS 100

Cite

@inproceedings{madaghiele2021mingus,
  author    = {Madaghiele, Vincenzo and Lisena, Pasquale and Troncy, Raphael},
  title     = {{MINGUS: Melodic Improvisation Neural Generator Using Seq2Seq}},
  booktitle = {22$^{nd}$ International Society for Music Information Retrieval Conference (ISMIR)},
  year      = {2021},
  month     = {11},
  address   = {Online}
}