Audio Processing Pipeline

Audio Processing Pipeline

What happens each frame to process audio. Two independent systems run in parallel:

1. PlayerAudioController — processes player voice audio
2. WorldAudioController — manages AudioSource emitters

Player Voice Pipeline

Frame Loop

PostLateUpdate (every frame)
  └─ For each remote player (time-sliced):
       ├─ 1. Get head positions
       ├─ 2. Find highest-priority override
       ├─ 3. Channel check (privacy)
       ├─ 4. Apply audio settings
       │    ├─ Occlusion (raycast)
       │    ├─ Directionality (dot products)
       │    ├─ Voice range reduction
       │    ├─ Avatar audio range reduction
       │    └─ Reverb
       └─ 5. Update AudioSources

Step-by-Step

1. Get Head Positions

• Listener: local player's camera position/rotation
• Emitter: remote player's head tracking data

2. Find Highest-Priority Override

• Check the player's PlayerAudioOverrideList
• GetMaxPriority() returns the override with highest priority
• If no override → use global configuration

3. Channel Check (Privacy)

• If emitter has a private channel and listener doesn't share it → mute
• If listener has DisallowListeningToChannel and emitter is outsider → mute
• See Private Voice Channels for details

4. Apply Audio Settings

Occlusion

• Raycast from listener head toward emitter head
• 0 hits → full clarity (1.0)
• 1 hit → check distance, apply obstacle/player clarity
• 2 hits → environment dominates, unless both hits are player avatars → player clarity applies
• See Occlusion for details

Directionality

• Two dot products: listener's forward vs direction-to-speaker, speaker's backward vs direction-to-listener
• Listener facing away → reduced range
• Speaker facing away → reduced range
• Both multiplied for final multiplier
• See Voice Directionality for details

Combined Multiplier

relativeAudioRange = occlusionClarity × directionalityMultiplier

Voice Range Reduction

voiceDistanceFar × heightMultiplier × relativeAudioRange
voiceDistanceNear × heightMultiplier × relativeAudioRange

Applied via VRCPlayerApi.SetVoiceDistanceFar() and SetVoiceDistanceNear().

Avatar Audio Range Reduction

Same calculation for avatar audio sources (footsteps, animations, etc.).

5. Update AudioSources

• SetVoiceGain(), SetVoiceVolumetricRadius()
• Avatar audio: SetAvatarAudioGain(), SetAvatarAudioFarRadius(), etc.
• Reverb filter applied if configured

Time-Sliced Processing

Not all players are updated every frame. PlayerUpdateRate controls how many:

Rate	80 players, 20fps	Full update time
0	All every frame	0.05s
1	1 per frame	4s
4	4 per frame	1s
80	All per frame	0.05s

Lower rate = better performance, less responsive. Higher rate = more responsive, more CPU.

World Audio Pipeline

Manages AudioSource emitters (music, ambient, sound effects) via importance-based prioritization.

See WorldAudio-Subsystem for full details.

Frame Loop

PostLateUpdate (every frame)
  ├─ 1. Rebuild registry if emitters changed
  ├─ 2. Recalculate importance (time-sliced)
  │    └─ For each emitter in batch:
  │         └─ RefreshInTree → AVL tree re-sort
  └─ 3. Update emitters (partition-based time-sliced)
       └─ For each emitter in batch:
            ├─ a. Privacy channel check
            ├─ b. Occlusion clarity (raycast)
            ├─ c. Culling (if beyond MaxActiveSources)
            └─ d. Apply reverb + lowpass filters

Step-by-Step

1. Rebuild Registry

When EmitterRegistry.NeedsRebuild is true (emitter added/removed):

• Rebuild Emitters[] array from dictionary
• Reinitialize partitions and update scheduler

2. Importance Recalculation

Each frame, recalculate importance for ImportanceUpdatesPerFrame emitters:

• Calculate importance based on distance, volume, priority, rolloff mode
• Remove from AVL tree, re-add with updated score
• Privacy-culled emitters are pushed behind the MaxActiveSources boundary

3. Emitter Updates

Each frame, update UpdatesPerFrame emitters via partition cursor:

Step	Description
Privacy check	Is this emitter audible to the local player?
Occlusion	Raycast from listener to emitter, calculate clarity
Culling	If emitter index ≥ MaxActiveSources, suspend it
Apply effects	Update reverb, lowpass filter, spatial blend

Partition-Based Time-Slicing

Emitters are divided into partitions with exponential bucket sizes:

Partition 0: [1 emitter]     ← highest priority, updates every frame
Partition 1: [4 emitters]    ← updates frequently
Partition 2: [16 emitters]   ← updates less often

Configure via PartitionState:

• Partitions (1–10) — number of buckets
• PartitionGrowthFactor (default: 4) — size multiplier between partitions (recommended range: 2–6)

See Also

• Architecture — component overview
• WorldAudio-Subsystem — importance-based emitter management
• PlayerAudioController — player voice controller
• WorldAudioController — emitter update controller