Symphony

Adaptive procedural music, a step sequencer, arrangement playback and export tools for Unity.

Symphony provides:

• A global tempo clock and bar/beat scheduling (SymphonyManager)
• A manual step sequencer with per-track options, swing, humanize, per-track speed multipliers
• A simple runtime synth and a plugin-based synth architecture
• Procedural melody and chord progression generation with style presets
• Arrangement playback across multiple patterns with optional tempo maps and per-clip audio overrides
• Audio FX chain system with a mixer and per-voice processing
• Robust editor tooling (Studio, Piano Roll, Song Builder/Arrangement, Browser, Plugin/Sound Engines, Mixer Console, Audio Settings, Exporter)
• Offline export to WAV/MIDI, with stems and optional mixer FX (OGG/MP3 via plugins)

This README documents the provided scripts and how to use them together.

---

Requirements

• Unity 2020.3+ (tested on recent LTS versions)
• A scene with a SymphonyManager or allow EnsureInstance to create one at runtime
• For offline export of OGG/MP3: integrate encoders (Vorbis/LAME) and define OGG_ENCODER_PRESENT / MP3_ENCODER_PRESENT

---

Core Components

• SymphonyManager
  • Global tempo clock, beat/bar scheduling, scheduleAheadTime
  • Methods: SecondsPerBeat, GetNextBeatDSPTime(), GetNextBarDSPTime()
• StepSequencerPattern
  • Tracks (Percussion/Synth), steps list, beats, stepsPerBeat, swingAmount, freeMove
  • TrackDef includes: defaultMidi, colorARGB, voiceColumns, volume, pan, mute, solo, mixerTrack, defaultProbability, defaultHumanizeMs, speedMultiplier
  • StepEvent includes: trackIndex, startBeat, durBeats, velocity, midi, sliceIndex, column, probability, humanizeMs
• ManualSequencerPlayer
  • Plays a StepSequencerPattern with slice/synth sources
  • Voice pool with prewarming to reduce runtime allocations
  • Scheduling with swing and humanize, per-track speed multiplier support
  • Trims clips to step ends/pattern ends; per-voice AudioFXChain routing to SymphonyMixer
  • Key fields: pattern, slicePack, synthPlugin, mixer, masterVolume, maxVoices, prewarmVoices
• SongArrangement
  • Clips list (ArrangementClip), beatsPerBar, tempoMap
  • ArrangementClip fields: pattern, startBeat, lengthBeats, laneIndex, name, muted
  • Per-clip overrides: slicePackOverride, synthOverride, mixerOverride
  • Utilities: NormalizeOrder(), ClipsAtBeat(), TotalBeats (max end of clips)
• ArrangementPlayer
  • Timeline-based playback across clips (uses ManualSequencerPlayer per clip)
  • Defaults: defaultSlicePack, slicePacks[], defaultSynth, defaultMixer
  • Playback options: regionTrimming, maxConcurrentClips, barSyncClipStart
  • Looping: loopEnabled, loopStartBeat, loopLengthBeats; dynamically recomputed per pass
  • CurrentPlaybackBeat exposes transport beat; Pause/Resume/Seek/Stop
  • Resolves SlicePack in priority: clip.slicePackOverride > defaultSlicePack > first non-null in slicePacks[]
  • Pooling: reuses clip players to reduce allocations and stutter
• SymphonyMixer
  • Channels (name, volume, pan) with per-channel FX chains (AudioFXDefinition)
  • IAudioEffect processors apply FX to audio buffers (runtime and export)
• SlicePack
  • Defines percussion slices and access to AudioClips
• SynthPluginBase (and custom plugins)
  • Provides CreateNoteClip for runtime generation; SampleInstrumentPlugin supported

---

Editor Tools

• Symphony Studio (Window → Symphony → Studio)
  • Step sequencer editing for patterns
  • Track controls (volume/pan/mute/solo/mixer channel)
  • Euclidean Fill & rhythm tools; trap hats (rolls); freeMove for off-grid placement
  • Slice palette for percussion; live playback/preview
  • Arrangement section shortcut; Export section
  • Multi-Player Preview: play all ManualSequencerPlayers in scene
• Piano Roll (Window → Symphony → Piano Roll)
  • Note editing for synth tracks: scales, presets, paste/marquee selection, snapping, resizing, velocity
• Song Builder (Window → Symphony → Song Builder)
  • FL-style Playlist with global timeline, lanes, snap, playhead
  • Pattern library drag-to-timeline; painting with selected sequences
  • Clip interactions: move, resize left/right, split at playhead, duplicate/delete, context menu
  • Transport with play/stop/reset; auto loop to content end (wraps at bounding box end)
  • Playback Sources: set ArrangementPlayer defaults (Default SlicePack, Slice Packs array, Default Synth, Default Mixer)
  • Live Add: painting clips while playing registers and starts them if appropriate
  • Keyboard shortcuts: Space for play/pause
• Arrangement Editor (Window → Symphony → Arrangement Editor)
  • Lightweight arrangement timeline editor with add clip panel, lanes, snapping
• Browser (Window → Symphony → Browser)
  • Asset browser with search and grouped categories; ping/select assets quickly
• Plugin Browser (Window → Symphony → Plugin Browser)
  • List SynthPluginBase assets, preview notes (C3/C4), open Sample Instrument editor
• Sound Engines (Window → Symphony → Sound Engines)
  • FL-style Generators window to manage Synth plugins and Slice packs
  • Preview notes/slices; assign defaults to ArrangementPlayer; append to slicePacks[]
• Mixer Console (Window → Symphony → Mixer Console)
  • FL-style Mixer window editing SymphonyMixer channels (name/volume/pan) and FX slots
  • Add/remove/reorder slots; assign AudioFXDefinition assets; apply as default mixer to ArrangementPlayer
• Audio Settings (Window → Symphony → Audio Settings)
  • View DSP buffer and output rate; apply quick presets (Stable/Balanced/Low-Latency)
  • Performance Settings asset integration: scheduleAheadTime, maxVoices, prewarmVoices; apply on play
• Exporter (Window → Symphony → Exporter)
  • Export Pattern to WAV/MIDI/OGG/MP3; options: sample rate, channels, normalization, dither (WAV), include percussion/synth
  • Shows export progress; auto refresh AssetDatabase for project folder outputs
  • Export Arrangement: supported in SymphonyExporter with stems and optional mixer FX

---

Playback Workflow

• Pattern Playback

  • Create a StepSequencerPattern or use Studio to design one
  • Add ManualSequencerPlayer to a scene, assign pattern, slice pack (for Percussion), synth plugin (for Synth), and optional mixer
  • Call PlayPattern(); StopPattern() to stop
  • ManualSequencerPlayer prewarms voices and schedules notes with swing/humanize

• Arrangement Playback

  • Create a SongArrangement asset; add ArrangementPlayer component to a GameObject
  • In Song Builder, assign Arrangement and Player, then drag patterns to lanes with beats
  • Set defaults: defaultSlicePack, slicePacks[], defaultSynth, defaultMixer
  • Optional per-clip overrides: slicePackOverride, synthOverride, mixerOverride
  • Looping
    • Auto loop to content end: Song Builder can set loopEnabled and loopLengthBeats to the bounding box end dynamically
    • ArrangementPlayer recomputes loop length/start each pass; playback will wrap at content end (ignores muted clips)
  • Bar-synced start: optionally align clip starts to next bar for transport continuity

---

Exporting

• Pattern export

  • SymphonyExporter.ExportPattern(pattern, slicePack, synth, mixer?, opts, out message)
  • Supports WAV/MIDI; OGG/MP3 when you integrate encoders and define symbols
  • Options: sampleRate, channels, bitDepth (WAV), normalize, dither, includePercussion/Synth, probability/humanize control

• Arrangement export

  • SymphonyExporter.ExportArrangement(arrangement, slicePack, synth, mixer?, opts, out message)
  • Computes total duration from clips and tempo map, renders stems per track, optionally applies mixer FX and sums into final mix
  • Equal-power pan law used for stereo balance; final normalization optional
  • Ensure SlicePack clips are readable (AudioClip Load Type: Decompress On Load) for offline export

• Encoders

  • OGG/MP3 require integrating libvorbis/lame or similar and wrapping in a plugin/DLL; define OGG_ENCODER_PRESENT/MP3_ENCODER_PRESENT and implement corresponding export functions

---

Performance & Stability

• Audio stutter prevention

  • Use SymphonyPerformanceSettings asset (Create → Symphony → Settings → Performance)
  • Apply DSP buffer presets (e.g., 512 samples × 4 buffers in Editor)
  • Use scheduleAheadTime in SymphonyManager (e.g., 0.15–0.25s) for safe scheduling
  • Limit maxConcurrentClips in ArrangementPlayer for very dense arrangements
  • ManualSequencerPlayer: leverage voice pooling and prewarm voices (prewarmVoices) to reduce runtime allocations
  • Avoid creating/destroying GameObjects and AudioSources per note; use pooling (implemented)
  • Use equal-power pan and normalization for balanced export mixes
  • Tune per-track speedMultiplier; extreme values increase scheduling load

• Known best practices

  • Keep schedule times slightly in the future (kScheduleLookaheadSec ~ 50ms) to avoid scheduling in the past
  • Prefer SlicePack clips with small tails; use trimClipsToStepEnd to cut long samples
  • If using mixer FX, ensure FX processors are efficient and avoid heavy allocations per Process call

---

Quick Start

1. Add SymphonyManager to your scene (or let EnsureInstance create one automatically via SymphonyPerformanceBootstrap).
2. Create a StepSequencerPattern (Assets → Create → Symphony → Step Sequencer Pattern) and edit it in Studio.
3. Add ManualSequencerPlayer to a GameObject; assign pattern, slicePack (for percussion), synthPlugin (for synth), and optional SymphonyMixer.
4. Press Play and call:

   var player = FindObjectOfType<ManualSequencerPlayer>();
   player.PlayPattern();

5. Build an arrangement:
   • Create a SongArrangement (Assets → Create → Symphony → Arrangement)
   • Window → Symphony → Song Builder; assign arrangement and create/find ArrangementPlayer
   • Drag patterns onto timeline lanes; press Play From Cursor
6. Loop to content end:
   • Toggle “Auto Loop To Content End” in Song Builder (transport); playback wraps at arrangement bounding-box end
7. Export:
   • Window → Symphony → Exporter; choose format and options; Export pattern
   • Or call ExportArrangement with mixer FX to render full arrangement

---

API Cheat Sheet

• SymphonyManager
  • double SecondsPerBeat { get; }
  • double GetNextBeatDSPTime(), GetNextBarDSPTime()
  • double scheduleAheadTime; double bpm
• ManualSequencerPlayer
  • pattern, slicePack, synthPlugin, mixer, masterVolume
  • int maxVoices, int prewarmVoices
  • void PlayPattern(), void StopPattern(), bool isPlaying
• SongArrangement
  • int beatsPerBar; List clips; TempoMap tempoMap
  • void NormalizeOrder(); IEnumerable ClipsAtBeat(float beat)
  • float TotalBeats { get; } (max end of clips)
• ArrangementPlayer
  • arrangement; defaultSlicePack; SlicePack[] slicePacks; defaultSynth; defaultMixer
  • bool regionTrimming; int maxConcurrentClips
  • bool loopEnabled; float loopStartBeat; float loopLengthBeats; bool barSyncClipStart
  • bool IsPlaying; float CurrentPlaybackBeat
  • void PlayArrangementFromBeat(float beat); void StopArrangement(); void PauseArrangement(); void ResumeArrangement(); void SeekToBeat(float beat)
  • void RegisterLiveClip(ArrangementClip clip)
• SymphonyMixer
  • List channels
  • MixerChannel: name, volume, pan, List fx
• SymphonyExporter
  • bool ExportPattern(pattern, slicePack, synth, mixer?, opts, out message)
  • bool ExportArrangement(arrangement, slicePack, synth, mixer?, opts, out message)
  • ExportOptions: format, filePath, sampleRate, channels, bitDepth, normalize, dither, includePercussion, includeSynth, stems, mixer FX, swing/probability options, etc.

---

Advanced Topics

• Tempo Map
  • SongArrangement.tempoMap allows BPM changes across beats; ExportArrangement maps beats to seconds across changes
• Per-clip overrides
  • ArrangementClip supports slicePackOverride, synthOverride, mixerOverride for targeted routing
• Automation
  • Future: hook up AutomationEnvelope assets for parameter automation in sequencer/mixer
• Sends/Aux
  • Consider extending SymphonyMixer with sends and bus channels for reverb/chorus
• Visual Scripting / PlayMaker
  • Provide units/actions to trigger PlayPattern, PlayArrangementFromBeat, SetTempo

---

Troubleshooting

• “No sound” or NullReference in ManualSequencerPlayer.SchedulePattern

  • Ensure pattern is assigned and pattern.tracks/steps are not null
  • Assign slicePack for percussion tracks and synthPlugin for synth tracks
  • Verify SymphonyManager exists in scene; enter Play Mode
  • Check AudioClips are readable (Load Type: Decompress On Load) for offline export

• Audio stuttering

  • Increase scheduleAheadTime (Audio Settings window)
  • Use a larger DSP buffer (e.g., 512–1024 × 4 buffers) in Editor
  • Enable voice pooling (prewarm voices) and avoid per-note object creation
  • Limit maxConcurrentClips in ArrangementPlayer

• Loop length wrong / wraps too early

  • In Song Builder, enable “Auto Loop To Content End”
  • Ensure muted clips are not considered; ArrangementPlayer recomputes loop window each pass

• Export issues

  • For stems and mixer FX application, ensure mixer channels and FX definitions are correctly assigned
  • Normalize final mix if clipping occurs; dither for PCM16 WAV to reduce quantization noise
  • OGG/MP3 require external encoders; integrate DLLs and define the relevant symbols

---

File/Folder Map (selected)

• Scripts/Core: SymphonyManager
• Scripts/Sequencer: StepSequencerPattern, TrackDef, StepEvent
• Scripts/Sampling: SlicePack, SampleSlice
• Scripts/Synth: SynthPluginBase, SimpleSynthPlugin, SampleInstrumentPlugin
• Scripts/Playback: ManualSequencerPlayer, ArrangementPlayer, ProceduralMusicPlayer, SimpleSynth
• Scripts/Mixer: SymphonyMixer, AudioFXDefinition, IAudioEffect, AudioFXChain (runtime)
• Scripts/Arrangement: SongArrangement
• Scripts/Tempo: TempoMap
• Scripts/Export: SymphonyExporter, MidiWriter
• Scripts/Settings: SymphonyPerformanceSettings
• Scripts/Runtime: SymphonyPerformanceBootstrap
• Scripts/Editor: Studio, Piano Roll, Song Builder, Arrangement Editor, Browser, Plugin Browser, Sound Engines, Mixer Console, Audio Settings, Exporter, Themes (TechGUI)

---

License / Credits / TODO

• OGG/MP3 export placeholders: integrate encoders and define OGG_ENCODER_PRESENT / MP3_ENCODER_PRESENT.
• FX processors: ensure AudioFXDefinition implementations are optimized and tested across platforms.
• Automation & Sends: planned features for richer mixing and parameter control.
• Visual Scripting/PlayMaker: add units/actions under Scripts/Integration as needed.