Library Indexing

Library Indexing & File Watching

Relevant source files

The following files were used as context for generating this wiki page:

• daemon/src/file_watcher_thread.rs
• daemon/src/helpers/db.rs
• daemon/src/helpers/index.rs
• daemon/src/helpers/mod.rs

The Aurora daemon maintains a local metadata index of the user's music library to provide fast search and retrieval without parsing audio files on every request. This subsystem is responsible for walking the filesystem, extracting tags, persisting metadata to a SQLite database, and responding to real-time filesystem changes.

The Indexing Pipeline (build_index)

The build_index function is the core of the library management system. It synchronizes the state of the filesystem with the songs table in the SQLite database and produces an in-memory SongIndex (a HashMap<Uuid, SongMeta>).

Implementation Logic

1. Filesystem Scan: It uses WalkDir to recursively scan the ~/Music directory for supported extensions: mp3, flac, wav, ogg, and m4a daemon/src/helpers/index.rs:103-125.
2. Cache Loading: The entire songs table is loaded into a HashMap keyed by file path to facilitate fast comparison daemon/src/helpers/index.rs:131-160.
3. Mtime Validation: For every file found on disk, the system compares the filesystem modification time (mtime) against the cached value in the DB daemon/src/helpers/index.rs:184-188.
   • Cache Hit: If mtime matches, the metadata is reconstructed from the DB without opening the file daemon/src/helpers/index.rs:189-205.
   • Cache Miss/Update: If the file is new or modified, symphonia is used to probe the file and extract tags (Title, Artist, Duration) daemon/src/helpers/index.rs:209-214.
4. ID Generation: Aurora uses UUID v5 (Namespace URL) generated from the file's absolute path string to ensure stable, reproducible identifiers across rescans daemon/src/helpers/index.rs:210.
5. Database Synchronization:
   • Upsert: New or changed metadata is batched and written to the songs table daemon/src/helpers/db.rs:26-34.
   • Stale Deletion: Any entries in the database whose paths no longer exist on the filesystem are purged to keep the library clean.

Metadata Extraction (read_tags)

The read_tags helper utilizes the symphonia crate to open a MediaSourceStream daemon/src/helpers/index.rs:40. It attempts to retrieve the "title" and "artist" tags from the metadata revisions daemon/src/helpers/index.rs:70-80. Duration is calculated by multiplying the track's n_frames by its time_base daemon/src/helpers/index.rs:82-91.

Sources: daemon/src/helpers/index.rs:31-99, daemon/src/helpers/index.rs:101-214, daemon/src/helpers/db.rs:23-54

Data Flow: Filesystem to SQLite

The following diagram illustrates how metadata moves from a physical file into the rusqlite database and the in-memory SongIndex.

Indexing Data Flow

graph TD
    subgraph "Filesystem"
        FS["Audio Files (~/Music)"]
    end

    subgraph "aurora-daemon (helpers/index.rs)"
        WD["WalkDir Scan"]
        SYM["Symphonia read_tags()"]
        V5["UUID v5 Generation"]
        CACHE_CHECK{"mtime Match?"}
    end

    subgraph "SQLite (aurora.db)"
        DB_SONGS[("TABLE songs")]
    end

    FS --> WD
    WD --> CACHE_CHECK
    CACHE_CHECK -- "No / New" --> SYM
    SYM --> V5
    V5 --> DB_SONGS
    CACHE_CHECK -- "Yes" --> SONG_INDEX["SongIndex (HashMap)"]
    DB_SONGS --> SONG_INDEX

Loading

Sources: daemon/src/helpers/index.rs:101-160, daemon/src/helpers/db.rs:26-34

File Watcher Thread

The file_watcher_thread provides real-time synchronization. It monitors the ~/Music directory for any create, modify, or remove events using the notify crate.

Debouncing Mechanism

To prevent excessive disk I/O during batch operations (like moving a folder), the watcher implements a 2-second debounce daemon/src/file_watcher_thread.rs:32.

• When an event occurs, a pending flag is set to true and the last_event timestamp is updated daemon/src/file_watcher_thread.rs:40-43.
• The thread loop checks if pending is true and if at least 2 seconds have elapsed since the last event daemon/src/file_watcher_thread.rs:60.
• Once triggered, it calls build_index to perform a full incremental rescan and updates the StateStruct with the new index daemon/src/file_watcher_thread.rs:63-67.

Watcher Thread Logic

sequenceDiagram
    participant FS as Filesystem
    participant FW as file_watcher_thread
    participant HI as helpers::build_index
    participant ST as StateStruct

    FS->>FW: notify::Event (Create/Modify/Remove)
    Note over FW: Set pending = true<br/>Reset last_event timer
    FW->>FW: Wait 2s (Debounce)
    FW->>HI: build_index(music_dir, db)
    HI->>FW: New SongIndex
    FW->>ST: Update state.index
    Note over ST: UI receives updated<br/>library on next status

Loading

Sources: daemon/src/file_watcher_thread.rs:6-75, daemon/src/helpers/index.rs:101-102

Database Schema

The songs table is the primary source of truth for the library. It stores both the metadata and the filesystem attributes required for the cache-invalidation logic.

Column	Type	Description
id	TEXT	Primary Key (UUID v5)
path	TEXT	Absolute path to the file (Unique)
title	TEXT	Extracted song title
artists	TEXT	JSON array of artist names
dur_ms	INTEGER	Duration in milliseconds
mtime	INTEGER	Last modified time (Unix Epoch)
art_path	TEXT	Path to extracted/cached album art

Sources: daemon/src/helpers/db.rs:26-34, daemon/src/helpers/index.rs:22-29