Sonic

Audio-based music similarity search for your personal library. Find songs that actually sound alike — not just the same genre tag.

What It Does

Point Sonic at your music library and search for any song. It finds the most sonically similar tracks using three neural audio models fused via Reciprocal Rank Fusion (RRF), with handcrafted audio feature filtering to catch tempo and energy mismatches.

• Audio-first — similarity is computed from the actual audio, not metadata
• Three-model consensus — EffNet, MusiCNN, and CLAP each capture different aspects of similarity and are fused via RRF for robust results
• Self-hosted — your music never leaves your machine
• Lightweight — runs on a mini PC, ~90MB RAM, <5ms queries

Quick Start

Prerequisites

• Docker
• A music library (FLAC, MP3, or any FFmpeg-supported format)
• Python 3.12+ (for embedding generation)

1. Generate Embeddings

Before running Sonic, you need to compute embeddings for your music library. This is a one-time operation.

# Install dependencies
pip install essentia-tensorflow numpy

# Generate EffNet embeddings (primary model)
python scripts/embed.py \
  --model effnet \
  --music-dir /path/to/your/music \
  --output data/embeddings/effnet_embeddings.npz

# Generate MusiCNN embeddings (secondary)
python scripts/embed.py \
  --model musicnn \
  --music-dir /path/to/your/music \
  --output data/embeddings/musicnn_embeddings.npz

# Generate CLAP embeddings (tertiary)
# Requires: pip install laion-clap torch torchaudio
python scripts/embed.py \
  --model clap \
  --music-dir /path/to/your/music \
  --output data/embeddings/clap_embeddings.npz

Embedding generation speed depends on your hardware:

Model	CPU (per track)	GPU (per track)
EffNet	~0.5s	~0.1s
MusiCNN	~0.5s	~0.1s
CLAP	~2s	~0.1s

For a 10K track library on CPU, expect ~1-2 hours per model.

2. Run Sonic

docker compose up -d

Open http://localhost:8000 in your browser.

3. Configuration

All settings are via environment variables:

# docker-compose.yml
services:
  sonic:
    build: .
    volumes:
      - /path/to/your/music:/music:ro
    ports:
      - "8000:8000"
    environment:
      - SONIC_MUSIC_ROOT=/music

See Configuration for all options.

How It Works

Three-Model Fusion

Sonic uses three pre-trained audio models, each capturing different aspects of musical similarity:

Model	What It Captures	Dimensions
Discogs-EffNet	Artist style, genre taxonomy (400 Discogs labels)	1,280
MSD-MusiCNN	Timbral texture, instrumentation patterns	200
LAION-CLAP	Acoustic texture, recording characteristics	512

These models have near-zero overlap in their top results — they genuinely measure different things. Sonic fuses them using Reciprocal Rank Fusion (RRF), which merges rank positions rather than raw scores:

rrf_score(track) = Σ 1/(k + rank_in_model)  for each model

This avoids the normalization problems of score-level fusion.

Post-Processing Pipeline

After RRF fusion, results pass through:

1. Near-duplicate removal — same recording across albums (cosine > 0.97)
2. Feature mismatch penalty — demotes tracks with mismatched tempo, energy, or spectral brightness (computed via librosa)
3. Artist exclusion — optional toggle to hide same-artist results
4. Consensus highlighting — tracks found by all three models get a gold star; tracks found by two get a blue dot

Audio Streaming

FLAC/MP3 files are transcoded to Opus on the fly via FFmpeg. No pre-processing needed — just point Sonic at your library.

Architecture

src/sonic/
├── app.py       # FastAPI routes (API + HTMX)
├── config.py    # Settings via environment variables
├── models.py    # Pydantic response types
├── search.py    # Three-model RRF search engine
├── library.py   # Track metadata + text search
├── stream.py    # FFmpeg audio transcoding
├── templates/   # HTMX server-rendered UI
└── static/      # Minimal client-side JS

Single Docker container. No external dependencies (no database, no vector DB, no cache). Embeddings loaded into numpy arrays at startup. Queries answered via brute-force cosine similarity in <5ms for libraries up to 100K tracks.

Configuration

Variable	Default	Description
SONIC_MUSIC_ROOT	/music	Path to music library (read-only)
SONIC_EMBEDDINGS_DIR	data/embeddings	Path to embedding files
SONIC_PRIMARY_MODEL	effnet	Primary retrieval model
SONIC_SECONDARY_MODELS	musicnn:musicnn_embeddings.npz,clap:clap_embeddings.npz	Secondary models for RRF
SONIC_FEATURES_FILE	audio_features.npz	Handcrafted features file
SONIC_RRF_TOP_N	50	Candidates per model for RRF
SONIC_DEDUP_THRESHOLD	0.97	Cosine threshold for duplicate detection
SONIC_DEFAULT_RESULT_LIMIT	10	Results per query
SONIC_STREAM_FORMAT	opus	Audio format (opus or mp3)
SONIC_STREAM_BITRATE	128k	Transcode bitrate
SONIC_ROOT_PATH	``	URL prefix for reverse proxy

API

GET /api/status                          → health check
GET /api/songs/search?q=miles+davis      → text search
GET /api/songs/{id}/similar              → find similar tracks
GET /api/songs/{id}/similar?exclude_artist=true → exclude same artist
GET /api/stream/{id}                     → audio stream

Generating Audio Features (Optional)

For tempo/energy mismatch filtering, generate handcrafted features:

pip install librosa
python scripts/extract_features.py \
  --music-dir /path/to/your/music \
  --output data/embeddings/audio_features.npz

This extracts tempo, RMS energy, and spectral centroid for each track (~0.5s per track on CPU).

Tech Stack

• Backend: Python 3.12, FastAPI, numpy
• Frontend: HTMX (server-rendered, no build step)
• Audio: FFmpeg (Opus/MP3 transcoding)
• Models: Essentia (EffNet, MusiCNN), LAION-CLAP
• Deployment: Docker

Contributing

Contributions welcome. Please open an issue first to discuss what you'd like to change.

License

MIT