Skip to content

Testing Protocol

Jump to bottom
Jakko edited this page May 8, 2026 · 1 revision

Testing Protocol

Setup

  1. Open eval/exercises/reaper-template.rpp. Kaiku is pre-instantiated on the MIDI track.
  2. Tempo: 40 BPM. REAPER ignores the MIDI file's embedded tempo. Set the project tempo manually before loading any eval MIDI. At 40 BPM: 6s sustain, 6s rest per note. At 120 BPM (REAPER default): 2s/2s — too short to evaluate decay or LFO tails.
  3. Gain staging: For ensemble exercises, set each track fader to −14 dB before rendering. Five voices at unity will clip (+14 dBTP).
  4. Enable the spectrum analyser on the master bus before rendering.
  5. Use headphones or nearfield monitors.

Render settings

  • Format: WAV, 48 kHz, 16-bit
  • Target peak: −5 to −6 dBFS sample peak
  • Check true peak (dBTP) in render stats — flag anything above 0 dBTP
  • Check integrated LUFS — solo reference is −6.1 LUFS; ensemble reference is −14.6 LUFS

What to listen for

The wheel beat

Stack C runs at 1.008× the fundamental. The slow amplitude modulation this produces is the instrument. It is not a defect. Do not tune it out. Do flag:

  • Beat that is inconsistent across registers (should slow monotonically from high to low)
  • Beat that disappears entirely on any note
  • Beat that is erratic or irregular in period

The reverb tail

The rest between notes is not silence — it is the test. After each note, listen for:

  • Clean exponential decay
  • No click or pop on release
  • Tail duration consistent with the register (low notes have more energy to decay)

The b2 (G♮)

The Phrygian blade. G natural is present in exercises E05, E07, E08, E09, E10, E11. If G♮ sounds wrong — thin, absent, or wrong in character relative to the other scale degrees — flag it. It is load-bearing for the instrument's identity.

Trompette

On accented notes (high velocity), a fourth transient voice fires and decays in ~80ms. The eval MIDIs use moderate velocity (80) and may not trigger it. Test separately with a high-velocity note after completing the eval suite. Confirm the transient fires cleanly and decays without artifact.

Spectrogram inspection (Marginal and Fail only)

After each render, view the WAV in REAPER's media browser with the spectrum analyser. Look for:

  • Clean harmonic series with consistent FM sidebands
  • Sideband pairs at ±1.008× intervals from each harmonic (the wheel carrier)
  • Flat noise floor
  • Aliasing appears as energy above 15 kHz — this is a defect
  • The LFO at 0.15 Hz is visible as slow amplitude periodicity on long notes

Do not attach spectrogram screenshots to passing exercises.

Reporting format

Record findings in Findings. One entry per exercise per session.

Fields:

  • Expected — what the exercise is designed to reveal
  • Observed — what was actually heard
  • Rating — Pass · Marginal · Fail
  • Notes — anything that doesn't fit the above

Marginal = something audible that cannot be characterized as pass or fail. Write what was heard.

Fail = artifact, wrong pitch, silence, pop, spectrogram anomaly.

Include with Marginal or Fail:

  • WAV render of the exercise
  • Spectrogram screenshot of the anomaly

Do not include:

  • REAPER project files
  • Spectrogram screenshots for passing exercises
  • Aesthetic opinions about the sound design
  • Corrected or revised MIDI

Clone this wiki locally