Chords with time to generate a midi file

[SutirthaChakraborty]

Hi,

I have a code that extracts the chords from the audio. It returns the following

1. The start time of the chord
2. Signature
3. Name of the chord
   (Image attached)
   

I want to recreate a MIDI file based on that information. How can I do that?

Thanks,
Suti

[jonathangjertsen]

This can be done by creating a ChordProgression object and calling its to_midi() method. You need to handle the case where the library doesn't understand the input string by catching the InvalidChord exception.

There is some example code in this unit test (requires jchord version 0.2.1): https://github.com/jonathangjertsen/jchord/blob/f3a63c949118e9261b761a5145073d12593be006/test/test_5_issues.py In your case the beats_per_chord list would be the differences between consecutive curr_beat values.

What did you use to extract the chords from audio? Depending on what kind of data it returns, it could be more efficient to use mido directly.

[SutirthaChakraborty]

Thank you for your example. I used MADMOM for extracting the chords. I think I have to individually change the chord names. Anything else you recommend?

[jonathangjertsen]

From looking at the documentation for that library, it seems like you could combine the note processor https://madmom.readthedocs.io/en/latest/modules/features/notes.html with the midi utilities https://madmom.readthedocs.io/en/latest/modules/io/midi.html to do the audio-to-midi conversion.

[SutirthaChakraborty]

Yes, exactly. Unfortunately, their chord notations are different than jchord.

[jonathangjertsen]

What I was thinking is that it could be possible to just pass the output of the RNNPianoNoteProcessor/NotePeakPickingProcessor into the MIDIFile.from_notes(...) function in madmom.

That said, the only difference seems to be that madmom adds a : between the root note and the chord quality. So in your image, prev_chord.replace(":", "") should return something that jchord recognizes.

[SutirthaChakraborty]

May be.. I will check that.

I can share you my code,

from madmom.features import CNNChordFeatureProcessor, CRFChordRecognitionProcessor, RNNDownBeatProcessor, DBNDownBeatTrackingProcessor

audio_file_name = "/content/Ride.mp3"

chord_processor = CNNChordFeatureProcessor()
chord_decoder = CRFChordRecognitionProcessor()
chords = chord_decoder(chord_processor(audio_file_name))

beat_processor = RNNDownBeatProcessor()
beat_decoder = DBNDownBeatTrackingProcessor(beats_per_bar=[4], fps=100)
beats = beat_decoder(beat_processor(audio_file_name))

chord_idx = 0
for beat_idx in range(len(beats) - 1):
  curr_beat_time, curr_beat = beats[beat_idx]

  # visually separate measures
  if int(curr_beat) == 1:
    print()

  # find the corresponding chord for this beat
  while chord_idx < len(chords):
    chord_time, _ , _= chords[chord_idx]
    prev_beat_time, _ = (0, 0) if beat_idx == 0 else beats[beat_idx - 1]
    eps = (curr_beat_time - prev_beat_time) / 2
    if chord_time > curr_beat_time + eps:
      break
    chord_idx += 1

  # print beat and chord info
  _, _, prev_chord = chords[chord_idx - 1]
  print(curr_beat_time, curr_beat, prev_chord)

[jonathangjertsen]

I got something reasonable by putting that code in a notebook and adding a new cell with this (using some other piano-based audio files):

from jchord.progressions import ChordProgression
from jchord.chords import ChordWithRoot, InvalidChord

chord_objects = []
durations = []
for start, stop, name in chords:
    name = name.replace(":", "")
    try:
        chord = ChordWithRoot.from_name(name)
    except InvalidChord:
        chord = ChordProgression.DUMMY_CHORD
    chord_objects.append(chord)
    durations.append(stop - start)

ChordProgression(chord_objects).to_midi("out.mid", beats_per_chord=durations)

[SutirthaChakraborty]

That works perfectly. But did you notice a delay(may be half beat) in your case?

Colab link

Too many experiments with code. sorry about that. See the bottom.
It seems okay if I use strings. But not with piano. It's with the chord recognition code.
Thank you for all your help.. Happy new year.

[jonathangjertsen]

The main issue here seems to be solved, created #71 for the issue with delay in generated midi files