Timeline and Waveform Editor

Timeline & Waveform Editor

Section: Desktop App · Desktop-UI-Architecture · Home · Media-Playback-System

The Timeline & Waveform Editor is the desktop app workstation surface for media alignment: duration-based peak buckets, multi-track timeline, transcript segment edits, dub review against source, and speaker tracks with synchronized scrubbing.

Architecture and Components

The editor follows a strict separation between domain logic, playback services, and UI presentation. The AvaloniaProjectDisplayBuilder acts as a coordination layer, transforming domain entities into view models suitable for the Avalonia UI shell.

Core Data Structures

The timeline is composed of multiple tracks and clips. Each clip represents a temporal segment of media, such as a source video or a generated dub.

Class/Record	Purpose
TimelineTrackViewModel	Represents a horizontal lane in the editor (e.g., Source, Ambiance, or Speaker).
TimelineClipViewModel	A discrete block of media within a track defined by start and end seconds.
WaveformSummary	Contains visualization data derived from normalized PCM audio.
WaveformSegmentBoundary	Defines the temporal bounds of a transcript segment for waveform highlighting.

Component Relationship

The following diagram illustrates the relationship between the project builders and the timeline components.

classDiagram
    class AvaloniaProjectDisplayBuilder {
        +BuildTimelineTracks()
        +BuildWaveformSegmentBoundaries()
    }
    class TimelineTrackViewModel {
        +string Name
        +AppColor Color
        +ObservableCollection~TimelineClipViewModel~ Clips
    }
    class TimelineClipViewModel {
        +double StartSeconds
        +double EndSeconds
        +string Label
        +WaveformSummary Waveform
    }
    class WaveformSummary {
        +IReadOnlyList~float~ Peaks
        +double DurationSeconds
    }

    AvaloniaProjectDisplayBuilder ..> TimelineTrackViewModel : creates
    TimelineTrackViewModel "1" *-- "many" TimelineClipViewModel : contains
    TimelineClipViewModel o-- WaveformSummary : displays

Loading

Waveform Visualization Logic

Waveform generation is performed from normalized PCM audio data. The editor uses "peak buckets" to maintain performance during rendering while preserving a consistent visual contract.

Layout Calculation

The WaveformLayout.Build function calculates the specific coordinates for rendering a waveform on a canvas. It maps audio peaks to visual bars based on the available width and height.

1. Peak Sampling: The system determines how many bars to render based on the canvas width.
2. Amplitude Mapping: It iterates through the WaveformSummary peaks, calculating the maximum amplitude for each bucket to determine bar height.
3. Temporal Mapping: It translates time-based data (playback position and segment boundaries) into pixel-based coordinates.

flowchart TD
    Start[Request Waveform Layout] --> GetWaveform[Get WaveformSummary & Segments]
    GetWaveform --> CalcBars[Calculate renderBarCount based on Width]
    CalcBars --> Loop[Iterate through Bar Indices]
    Loop --> FindMax[Find Max Amplitude in Peak Bucket]
    FindMax --> BuildBar[Create WaveformBarLayout]
    BuildBar --> Next{More Bars?}
    Next -- Yes --> Loop
    Next -- No --> Markers[Map Time to Pixels for Markers & Cursor]
    Markers --> Final[Return WaveformCanvasLayout]

Loading

Time-to-Pixel Mapping

The WaveformMapping utility provides the logic for bidirectional conversion between timeline time and UI coordinates.

• TimeToPixel: (timeSeconds / durationSeconds) * width (clamped 0.0 to 1.0).
• PixelToTime: (pixel / width) * durationSeconds.

Timeline Clip Management

The editor enforces strict rules for clip placement to prevent overlaps and ensure data integrity.

Placement Validation

The UiHelpers.CanPlaceClip method validates whether a new clip can be added to a track without conflicting with existing clips. It checks the proposed start and duration against the current collection of clips.

// Example placement validation from tests
(double Start, double Duration)[] existing = track.Clips
    .Select(clip => (clip.StartSeconds, clip.DurationSeconds))
    .ToArray();

bool canPlace = UiHelpers.CanPlaceClip(newStart, newDuration, existing);

Timeline Initialization

The BuildTimelineTracks method dynamically constructs the editor's visual lanes based on project state:

1. Source Track: Added if media is present, displaying the full source media waveform.
2. Ambiance Track: Added if an ambiance stem (Music + SFX) is available.
3. Speaker Tracks: Individual tracks for each detected speaker, containing clips for both original dialogue segments and generated dubs.

User Interface Mockup

The shell redesign mockup defines the intended UX for the timeline dock.

• Timeline Bar: Contains tools for Splitting (S), Merging, Trimming, and Deleting (Del) clips, along with zoom controls and a timecode display.
• Track Headers: Displays speaker names, colors (from the SpeakerPalette), and Mute/Solo controls.
• Visual Feedback: A playhead (vertical line) indicates the current playback position across all tracks.

Summary

The Timeline & Waveform Editor provides a robust framework for managing temporal media data within Trackdub. By utilizing peak-based waveform summaries and a multi-track view model architecture, it enables precise alignment and review of dubbing artifacts. The editor's reliance on the WaveformMapping logic ensures that visual representation remains perfectly synchronized with the underlying audio and transcript data.