Skip to content

Audio Processing

Ryan Lauterbach edited this page Jul 23, 2026 · 3 revisions

Audio processing

The Audio bounded context turns a direct-input guitar recording into one rendered segment per signal chain. It is implemented in model/audio and consumed by audio_worker.

Architectural boundary

Core defines the AudioProcessor port and the domain values that cross the boundary. Audio implements the port with digital signal processing libraries.

flowchart LR
    Core["Core<br/>AudioProcessor port and domain values"]
    Audio["Audio context<br/>PedalboardAudioProcessor and ChainExecutor"]
    Libraries["Third-party processing<br/>Neural Amp Modeler, Pedalboard and PyLoudnorm"]

    Audio --> Core
    Audio --> Libraries
Loading

The dependency points towards Core. The gts package never imports audio, PyTorch, Pedalboard or PyLoudnorm. import-linter prevents Audio from importing web applications, workers, source contexts or Video.

This is the ports-and-adapters boundary in concrete form:

  • AudioProcessor is the port.
  • PedalboardAudioProcessor is an adapter.
  • audio_worker is an inbound command consumer and application host.
  • The filesystem and gear-path resolver are outbound infrastructure.

Source layout

Module Responsibility
processing/processor.py Implements the general AudioProcessor port.
processing/chain_executor.py Executes a domain SignalChain block by block.
processing/nam_loader.py Loads and caches Neural Amp Modeler files.
processing/ir_loader.py Validates and loads cabinet impulse responses.
processing/loudness.py Measures and normalises EBU R128 loudness.
processing/permutation.py Expands controlled signal-chain alternatives.
analysis/waveform.py Extracts a display envelope for the player.

Domain model

A SignalChain is an aggregate. Its blocks have an explicit position and gear type. The domain model and SignalChainValidator define the legal grammar before processing starts.

Current block positions are pre-effect, amplifier, effects loop, impulse response and post-effect. The processing order is the numeric block position.

The important constraints are:

  • A head capture (AMP) requires an impulse response for cabinet simulation.
  • A full-rig capture (FULL_RIG) cannot also use an impulse response because its cabinet is already present.
  • An empty chain cannot render.
  • Unsupported gear types fail with a processing error.

The browser does not invent these rules. It consumes the same domain vocabulary and API guidance used by backend services.

Rendering flow

flowchart TD
    DI["Load DI audio"]
    Mono["Convert stereo input to mono"]
    Resample["Resample when required"]
    Blocks["Execute ordered signal-chain blocks"]
    NAM["Run NAM model blocks"]
    IR["Convolve cabinet impulse response"]
    Loudness["Normalise to -14 LUFS"]
    File["Write version-scoped WAV segment"]
    Waveform["Extract waveform and metrics"]

    DI --> Mono --> Resample --> Blocks
    Blocks --> NAM
    NAM --> IR
    IR --> Loudness --> File --> Waveform
Loading

NAM inference runs through the upstream neural-amp-modeler package. GTS loads and executes models; it does not train them. NAM blocks use cached model instances. Impulse responses run through Pedalboard convolution.

The worker runs CPU-bound processing off the asynchronous event loop. Its message lease heartbeat continues while the render runs, so a long render does not look abandoned.

Shootout processing

The shootout orchestrator creates one SHOOTOUT_AUDIO child job per chain. The audio worker:

  1. Loads the job, shootout, DI track and hydrated signal chain.
  2. Resolves each model or impulse response through controlled storage.
  3. Runs the chain against the shared DI.
  4. Normalises the result and writes it beneath the shootout render version.
  5. Extracts waveform and loudness data.
  6. Inserts the AudioSegment and completes the child job.
  7. Calls reconciliation so the parent can advance or fail.

Every chain uses the same source recording from time zero. The per-chain segments are therefore the basis for the A/B comparison. The optional master.wav is a sequential montage for download or later enrichment; the comparison player does not use it.

Output contract

An audio result records:

  • Output path inside the trusted processing boundary.
  • Duration and sample rate.
  • Peak level in dBFS.
  • Integrated loudness in LUFS.
  • Processing time.

Storage paths never cross the public API. Finalisation converts the internal result into an opaque media reference in an immutable shootout manifest.

The render writes WAV output. The public media contract and range-request behaviour are owned by the web application, not the Audio context.

Failure model

Audio code fails explicitly for missing or invalid NAM files, invalid impulse responses, silent audio, illegal chains and unsupported gear types. The worker translates processing failures into job state and reconciliation.

Message delivery is at least once. Job and segment identity therefore make a repeated delivery safe. The worker must not overwrite a published render version or complete a job twice.

Current maturity

The processing library, chain execution, worker command path, segment persistence, montage implementation and finalisation integration exist in the repository. Issue #157 remains the delivery record for validating the complete user-triggered flow and its failure cases.

References

Clone this wiki locally