Music Routing Transformer Colab

[asigalov61]

Hey guys,

I had a great experience with this implementation of the Routing Transformer thanks to the efforts of @lucidrains so I wanted to share with you my creation which is based on this code/repo/implementation.

Here is the link:

https://github.com/asigalov61/Music-Transformers-Library/tree/main/Routing-Transformer

I really liked that this RT trains well and quickly. And I really enjoyed the fast generation speeds and the quality of the output.

Thank you and I hope you may find my Colab interesting and useful.

Thanks.

Alex

P.S. @lucidrains GitHub has a new feature: Discussion Boards for the repos, so I would suggest enabling it everywhere so that things like creations could be shared separately from the Issues. Just my humble suggestion. Thanks.

[asigalov61]

@lucidrains Here are the results of the decent training run IMHO. Works great!

NUM_BATCHES = int(1e5)
BATCH_SIZE = 8
GRADIENT_ACCUMULATE_EVERY = 4
LEARNING_RATE = 3e-4
VALIDATE_EVERY = 100
GENERATE_EVERY = 500
GENERATE_LENGTH = 512
SEQ_LEN = 4096

training: 4%|▎ | 3582/100000 [3:55:41<104:12:46, 3.89s/it]training loss: 0.12034496665000916

Generation 2048 @ 0.6-0.8 temp...

See attached MIDI samples.

Thanks.

Music-Routing-Transformer_MIDI_Output_Samples.zip