Media Ingest and Preparation

Media Ingest & Preparation

Section: Pipeline · Pipeline-Orchestration · Home · Transcription-and-VAD

Media Ingest & Preparation brings local media into a project: register source media, probe metadata, and produce artifacts such as normalized audio and waveforms for downstream ASR/translation. Sources: README, LONGTERM-ROADMAP.

Local-first media spine: register source first; heavy work (audio extraction) is lazy. Sources: AGENT_CONTEXT, ProjectMediaIngestServiceTests.

Core Architecture & Components

The ingest system is primarily orchestrated by the ProjectMediaIngestService. This service coordinates between the filesystem, database repositories, and specialized media services to create a stable project environment.

Ingest Workflow

The ingest process follows a specific sequence to ensure data integrity and artifact preservation.

graph TD
    A[Source Media File] --> B[Media Probe]
    B --> C[Create Media Spine]
    C --> D[Audio Extraction]
    D --> E[Normalization]
    E --> F[Waveform Generation]
    F --> G[Project Ready]
    
    subgraph "Application Layer"
    C
    end
    
    subgraph "Media Layer"
    B
    D
    E
    F
    end

Loading

Logical flow from raw media to a prepared project state. Sources: README, LONGTERM-ROADMAP.

Key Service Methods

Method	Purpose	Key Artifacts
CreateAsync	Full project creation including immediate heavy ingest.	Normalized Audio, Waveform Summary
CreateMediaSpineAsync	Lightweight registration of source media without heavy processing.	Manifest, Source Reference
EnsureNormalizedAudioAsync	Lazy extraction of audio if it does not already exist.	Normalized Audio
RelocateSourceAsync	Updates the path of a project's source media if moved on disk.	Updated Media Asset
EnsureStereoNormalizedAudioAsync	Refreshes mono artifacts to stereo for stem separation.	Stereo Normalized Audio

Audio Preparation & Normalization

A critical part of preparation is creating a "Normalized Audio" artifact. This artifact serves as the primary input for VAD (Voice Activity Detection) and ASR (Automatic Speech Recognition) stages.

Normalization Logic

Normalization ensures consistent audio levels (typically targeting LUFS standards) and sample rates across different source media types. This is essential for the accuracy of AI models used later in the pipeline. Sources: README, LONGTERM-ROADMAP.

sequenceDiagram
    participant P as ProjectMediaIngestService
    participant E as IAudioExtractionService
    participant W as IWaveformSummaryGenerator
    participant S as IArtifactStore

    P->>E: ExtractNormalizedAudioAsync(sourcePath)
    E-->>P: AudioExtractionResult (WAV Path)
    P->>W: GenerateAsync(audioPath)
    W-->>P: WaveformSummary
    P->>S: WriteJsonAsync(WaveformSummary)
    Note over P, S: Artifacts Registered in Media Repository

Loading

Sequence of audio normalization and waveform generation during ingest. Sources: tests/Trackdub.Application.Tests/ProjectMediaIngestServiceTests.cs:228-260

Project Media Persistence

Trackdub uses a "one database per project" model using SQLite. Ingested media data and artifact metadata are stored within this database, while the actual binary files (like extracted WAVs) are stored on the filesystem relative to the project. Sources: AGENT_CONTEXT, Local-Project-Persistence.

Artifact Governance

Every artifact generated during preparation is recorded with specific metadata:

• Provenance: Records what process created the artifact (e.g., normalized-audio-refresh:media-spine).
• Artifact Kind: Categorizes the file (e.g., NormalizedAudio, WaveformSummary, StemSeparationSourceAudio).
• Technical Metadata: Includes duration, channel count, and sample rate.
• Integrity: Stores file hashes to detect corruption or changes.

Handling Missing or Relocated Media

Since Trackdub is local-first, it must handle scenarios where the original source media is moved or deleted. The ProjectMediaIngestService provides mechanisms to detect missing files and update project references.

Relocation Sequence

flowchart TD
    A[Open Project] --> B{Source Found?}
    B -- No --> C[Status: Missing]
    C --> D[User Triggers Relocate]
    D --> E[Fingerprint Check]
    E --> F[Update Media Asset Path]
    F --> G[Status: Available]
    B -- Yes --> G

Loading

Logic flow for handling missing source media and the relocation process. Sources: tests/Trackdub.Application.Tests/ProjectMediaIngestServiceTests.cs:134-180

Conclusion

Media Ingest & Preparation acts as the gatekeeper for the Trackdub pipeline. By separating the "Media Spine" (metadata) from the heavy artifacts (binary data), the system provides a responsive user experience while maintaining the strict artifact governance required for high-quality AI dubbing. Sources: LONGTERM-ROADMAP, AGENT_CONTEXT.