From Model Output to MIDI
Model inference produces arrays of scores and attributes. A2M must turn those arrays into a single ordered collection of notes and pedal changes.
For each modeled symbol, a backward dynamic program compares advancing through a skip with selecting one of the candidate intervals. A simplified recurrence is:
D(t) = max(D(t+1) + s_skip(t), maxᵤ>ₜ[s_interval(t,u) + D(u)])
D(t) is the best remaining path score from time index t. Back-pointers record which choice won so the selected intervals can be reconstructed. The figure's heatmap is explanatory rather than a dump of one particular inference tensor.
Decoded interval coordinates and scorer context enter the attribute model. If a frame index is f, the model predicts a bounded fractional adjustment δ ∈ [-0.5, 0.5]:
t = (f + δ) × 1024 / 44100
One frame spans about 23.22 ms, so a boundary can move by approximately ±11.61 ms. Each interval also receives a MIDI velocity class and onset/offset-presence values used at section edges.
Section-local positions are translated using the section start and leading context padding. Negative translated times are clamped to zero. Decoder state travels forward so an event crossing a boundary is not treated as entirely new.
When an event overlaps the latest event for the same symbol:
- a recognized new onset replaces the overlapping estimate;
- a continuation without a new onset extends or updates the prior estimate;
- only an event with a recognized onset begins a new output event.
After sorting, a later onset truncates an earlier overlapping note of the same pitch. Zero-duration events are removed, and remaining times are bounded to the real source duration. Consequently, the current representation cannot retain two independently sustained voices on exactly the same MIDI pitch.
With pedal export enabled, internal symbol -64 becomes sustain CC64 and -67 becomes soft-pedal CC67. Onsets write an active controller value and offsets write zero.
Expressive dynamics use the predicted velocity, clamped from 1 through 127. Uniform dynamics replace every note velocity with the selected value. Pedal values are not changed by the note dynamics setting.
A2M uses PrettyMIDI at 960 PPQ and creates one non-drum Acoustic Grand Piano instrument named A2M Piano. Notes and controllers are sorted before writing. A unique output name is reserved so an existing file is not intentionally overwritten; an incomplete reserved file is removed if conversion fails.
See Worked Example and Validation for the resulting events aligned with their waveform and log-frequency spectrogram.
Primary source: A2M/a2m/core/conversion_service.py and the decoding/stitching functions in piano_engine.py.