Skip to content

Architecture

Jump to bottom
“Jesse edited this page Apr 7, 2026 · 4 revisions

Architecture

This page covers the major subsystems in Stillwater, how they connect, and the key files that implement each one. For project structure and design principles, see the Developer Guide.

Bootstrap Sequence

The application starts in cmd/stillwater/main.go. The run() function initializes services in dependency order, then starts background goroutines and the HTTP server.

flowchart TD
    A[Load Config] --> B[Init Logging]
    B --> C[Open Database + WAL mode]
    C --> D[Run Goose Migrations]
    D --> E[Init Core Services]
    E --> F[Init Provider Infrastructure]
    F --> G[Init Rule Engine + Pipeline]
    G --> H[Init Event Bus + Webhooks]
    H --> I[Start Background Goroutines]
    I --> J[Start HTTP Server :1973]

    E --- E1[AuthService]
    E --- E2[ArtistService]
    E --- E3[PlatformService]
    E --- E4[ConnectionService]
    E --- E5[LibraryService]

    F --- F1[RateLimiterMap]
    F --- F2[Provider Registry]
    F --- F3[Orchestrator]

    I --- I1[Event Bus dispatcher]
    I --- I2[Backup scheduler]
    I --- I3[Maintenance scheduler]
    I --- I4[Session cleanup - hourly]
    I --- I5[Rule scheduler - opt-in]
    I --- I6[Filesystem watchers]
Loading

Dependency injection is handled through the RouterDeps struct, which aggregates all services and is passed to NewRouter(). There is no DI framework -- services are constructed in main.go and wired together explicitly.

Background goroutines started at boot:

  • Event bus (bus.Start()) -- drains the buffered channel and dispatches to subscribers
  • Backup scheduler -- runs at a configured interval (if enabled)
  • Maintenance scheduler -- reads interval from DB settings (default 24h)
  • Session cleanup -- hourly, removes expired sessions
  • Rule evaluation scheduler -- opt-in, configurable intervals (6/12/24/48h)
  • Filesystem watchers -- one per library, detects new/removed artist directories
  • HTTP server -- ListenAndServe in a goroutine with graceful shutdown

Request Lifecycle

Every HTTP request passes through a middleware chain before reaching a handler. The chain is applied in the order shown below.

sequenceDiagram
    participant C as Client
    participant S as SecurityHeaders
    participant L as Logging
    participant CS as CSRF
    participant A as Auth
    participant H as Handler

    C->>S: HTTP Request
    S->>L: Add security headers
    L->>CS: Log request start
    CS->>A: Validate CSRF token
    A->>H: Authenticate (session or API token)
    H-->>A: Response
    A-->>L: Pass through
    L-->>C: Log duration + status
Loading

Middleware chain order:

  1. SecurityHeaders -- Sets HSTS, CSP, X-Frame-Options, X-Content-Type-Options
  2. Logging -- Logs request method, path, status code, and duration via slog
  3. CSRF -- Validates CSRF token on state-changing requests (POST, PUT, DELETE). Exemptions: login/setup endpoints, requests authenticated via API token (Bearer header or query param)
  4. Auth -- Extracts and validates credentials. Sets user ID, auth method, and token scopes in the request context

Key files:

  • internal/api/router.go -- Route registration, RouterDeps struct
  • internal/api/middleware.go -- Middleware implementations
  • internal/api/middleware_auth.go -- Auth middleware and context helpers

API Layer

Routes are registered using Go 1.22+ net/http method-based routing in internal/api/router.go. Each route maps a method and path pattern to a handler method.

mux.HandleFunc("GET /api/v1/artists", h.ListArtists)
mux.HandleFunc("GET /api/v1/artists/{id}", h.GetArtist)
mux.HandleFunc("POST /api/v1/artists/{id}/fetch", h.FetchArtistMetadata)

Route categories:

  • Public: /api/v1/health, /api/v1/docs, login, setup, static files
  • Protected: All other endpoints (require Auth middleware)
  • Scoped: Webhook endpoints require RequireScope("webhook")

Response patterns:

  • API endpoints return JSON (application/json)
  • Web UI endpoints return HTML fragments for HTMX consumption
  • Shared handlers check the Accept header or use separate route paths

Authentication

Stillwater supports two authentication methods, both handled by the Auth middleware.

Session-based (web UI):

  • Sessions stored in the sessions table with 24-hour expiry
  • Session token: 32 random bytes, hex-encoded, stored in a session cookie
  • Passwords: SHA-256 pre-hash + bcrypt

API token (external clients):

  • Prefix: sw_ (identifies Stillwater tokens)
  • Format: sw_ + 32 random hex bytes
  • Storage: SHA-256 hash of the plaintext token in the database
  • Plaintext shown once at creation, never stored
  • Scopes: read, write, webhook, admin (admin grants all)
  • Last-used timestamp updated asynchronously

Token extraction order (in Auth middleware):

  1. session cookie (web UI)
  2. Authorization: Bearer ... header (API clients)
  3. apikey query parameter (webhook callback URLs)

Context keys set by the middleware:

  • userID -- authenticated user ID
  • authMethod -- "session" or "api_token"
  • tokenScopes -- comma-separated scope string

Helper functions (internal/api/middleware_auth.go):

  • UserIDFromContext(ctx) -- extract user ID
  • HasScope(ctx, scope) -- check if the context has a specific scope
  • RequireScope(scope) -- middleware factory that rejects requests without the given scope

Key files:

  • internal/auth/auth.go -- AuthService (session CRUD, password hashing, token generation)
  • internal/api/middleware_auth.go -- Auth middleware, context helpers, scope enforcement

Provider System

Providers are adapters that fetch artist metadata and images from external sources. The system supports twelve providers:

Provider Description Auth
MusicBrainz Community music metadata database Free, no key
Fanart.tv High-quality artist artwork Optional API key
TheAudioDB Audio metadata and images Optional API key
Discogs Music database with release data Optional API key
Last.fm Listening stats and artist info API key required
Wikidata Structured data from Wikimedia Free, no key
Deezer Streaming platform metadata Free, no key
DuckDuckGo Web image search Free, no key
Genius Lyrics and artist bios API key required
Wikipedia Artist biographies Free, no key
AllMusic Professional music reviews and bios Free, no key
Spotify Streaming platform metadata API key required 
flowchart TD
    O[Orchestrator] -->|priority order| R[Registry]
    R --> P1[MusicBrainz]
    R --> P2[Fanart.tv]
    R --> P3[TheAudioDB]
    R --> P4[Discogs]
    R --> P5[Last.fm]
    R --> P6[Wikidata]
    R --> P7[Deezer]
    R --> P8[DuckDuckGo]
    R --> P9[Genius]
    R --> P10[Wikipedia]
    R --> P11[AllMusic]
    R --> P12[Spotify]

    O -->|rate limit| RL[RateLimiterMap]
    RL --> RL1[1 req/sec per provider]

    O -->|field merge| PS[ProviderSettings]
    PS -->|priority config| O
Loading

Provider interface (internal/provider/provider.go):

type Provider interface {
    Name() ProviderName
    RequiresAuth() bool
    SearchArtist(ctx context.Context, name string) ([]ArtistSearchResult, error)
    GetArtist(ctx context.Context, id string) (*ArtistMetadata, error)
    GetImages(ctx context.Context, id string) ([]ImageResult, error)
}

A separate WebImageProvider interface adds SearchImages() for providers like DuckDuckGo that only support web-based image search.

Registry (internal/provider/registry.go):

  • Thread-safe map of ProviderName to Provider
  • Register(p) / Get(name) / All() (stable iteration order)

Orchestrator (internal/provider/orchestrator.go):

  • Coordinates multi-provider queries
  • FetchMetadata() -- queries providers in priority order, merges results using field-level priority settings
  • FetchImages() -- fetches from all image-capable providers
  • Search() -- searches all providers, deduplicates results
  • Priority-based field merging: the user configures which provider data wins for each field (biography, genres, etc.)

Rate limiting:

  • RateLimiterMap is a singleton created at startup in main.go
  • One rate.Limiter per provider, shared across all handlers and goroutines
  • MusicBrainz: 1 request/sec (enforced by their API policy)

Rule Engine

The rule engine evaluates artist metadata against configurable quality rules and optionally auto-fixes violations.

Core types (internal/rule/):

Type Purpose
Rule Definition stored in DB: name, category (nfo/image/metadata), automation mode, config
Checker Function that evaluates a single rule against an artist, returns a Violation or nil
Violation Transient result: severity, message, fixable flag
RuleViolation Persisted violation: status (open/dismissed/resolved/pending_choice), image candidates
Fixer Interface that can fix a specific violation type
Pipeline Orchestrates evaluation + auto-fixing across all artists

Built-in checkers (internal/rule/checkers.go):

  • NFO: NFOExists, NFOHasMBID
  • Thumbnail: ThumbExists, ThumbSquare, ThumbMinRes
  • Fanart: FanartExists, FanartMinRes, FanartAspect
  • Logo: LogoExists, LogoMinRes
  • Banner: BannerExists, BannerMinRes
  • Metadata: BioExists
  • Cleanup: ExtraneousImages

Built-in fixers (internal/rule/fixer.go):

  • NFOFixer -- restores NFO from snapshot
  • MetadataFixer -- fetches missing metadata from providers
  • ImageFixer -- fetches missing images
  • ExtraneousImagesFixer -- deletes extraneous files

Automation modes (per rule):

  • auto -- evaluate and auto-fix if the violation is fixable
  • manual -- evaluate but persist as open (user must review)
  • disabled -- skip entirely

Pipeline (internal/rule/fixer.go):

  • RunAll() -- evaluate all artists, attempt fixes per automation mode
  • RunRule(ruleID) -- single rule against all artists
  • RunForArtist(artist) -- all rules against one artist

Scheduler (internal/rule/scheduler.go):

  • Opt-in via DB setting (rule_schedule.interval_hours)
  • Intervals: 6, 12, 24, or 48 hours

Scanner

The scanner discovers artist directories in configured music libraries, detects image files, and parses NFO files.

Scan flow (internal/scanner/scanner.go):

  1. Enumerate all configured library paths
  2. For each library, list top-level subdirectories (one per artist)
  3. Skip excluded names (case-insensitive): "Various Artists", "Various", "VA", "Soundtrack", "OST"
  4. Detect image files using platform-aware filename patterns
  5. Parse artist.nfo if present
  6. Create or update artist records in the database
  7. Publish events (FSDirCreated, FSDirRemoved, ScanCompleted)

Image detection patterns:

Type Filenames
Thumbnail folder.jpg, folder.png, artist.jpg, artist.png, poster.jpg, poster.png
Fanart fanart.jpg, fanart.png, backdrop.jpg, backdrop.png
Logo logo.png, logo-white.png
Banner banner.jpg, banner.png

Concurrency: Only one scan runs at a time (mutex-protected). Run() returns immediately with a ScanResult ID; the scan executes asynchronously. Status() returns the current or most recent scan snapshot.

Database

Connection (internal/database/database.go):

  • Pure-Go SQLite via modernc.org/sqlite
  • WAL mode enabled (_journal_mode=WAL)
  • Foreign keys enabled (_foreign_keys=ON)
  • Busy timeout: 5000ms
  • MaxOpenConns: 1 (SQLite serializes writes; a single connection avoids lock contention)

Migrations:

  • Goose (SQL-based), located in internal/database/migrations/
  • Naming: YYYYMMDDHHMMSS_description.sql
  • Each file has -- +goose Up and -- +goose Down sections
  • Migrations run automatically on startup

Transaction batching strategy:

Batch size Strategy
< 100 items Single transaction
100-1000 Transactions of 50
1000+ Transactions of 25 with short sleep between batches

User-initiated actions get priority over background jobs.

Schema

Artist data is normalized across several tables. The artists table holds core metadata, while related data is split into dedicated tables linked by foreign keys.

erDiagram
    libraries ||--o{ artists : "has"
    artists ||--o{ artist_provider_ids : "identified by"
    artists ||--o{ artist_images : "has"
    artists ||--o{ artist_aliases : "has"
    artists ||--o{ band_members : "has"
    artists ||--o{ nfo_snapshots : "has"
    artists ||--o{ artist_platform_ids : "maps"
    connections ||--o{ artist_platform_ids : "maps"
    connections ||--o{ libraries : "owns"
    artists ||--o{ rule_violations : "checked by"
    rules ||--o{ rule_violations : "defines"

    artists {
        TEXT id PK
        TEXT name
        TEXT sort_name
        TEXT library_id FK
        REAL health_score
    }
    artist_provider_ids {
        TEXT artist_id PK
        TEXT provider PK
        TEXT provider_id
        TEXT fetched_at
    }
    artist_images {
        TEXT id PK
        TEXT artist_id FK
        TEXT image_type
        INTEGER slot_index
        INTEGER exists_flag
        INTEGER low_res
    }
    artist_aliases {
        TEXT id PK
        TEXT artist_id FK
        TEXT alias
        TEXT source
    }
    artist_platform_ids {
        TEXT artist_id PK
        TEXT connection_id PK
        TEXT platform_artist_id
    }
    band_members {
        TEXT id PK
        TEXT artist_id FK
        TEXT member_name
        TEXT member_mbid
    }
    nfo_snapshots {
        TEXT id PK
        TEXT artist_id FK
        TEXT content
    }
    connections {
        TEXT id PK
        TEXT name
        TEXT type
    }
    libraries {
        TEXT id PK
        TEXT name
        TEXT path
        TEXT connection_id FK
    }
Loading

Artist tables:

Table Purpose Key relationships
artists Core artist/composer records (name, type, gender, biography, health score) FK: library_id refs libraries(id)
artist_provider_ids Provider identity mappings (MusicBrainz, AudioDB, Discogs, Wikidata, Deezer, Spotify, Last.fm) Composite PK: (artist_id, provider). FK to artists(id) ON DELETE CASCADE
artist_images Per-slot image metadata (exists, low_res, placeholder, dimensions, phash) Unique: (artist_id, image_type, slot_index). FK to artists(id) ON DELETE CASCADE
artist_aliases Alternative names for search and deduplication FK: artist_id refs artists(id) ON DELETE CASCADE
artist_platform_ids Platform-specific ID mappings (Emby, Jellyfin, Lidarr) Composite PK: (artist_id, connection_id). FKs to artists(id) and connections(id)
band_members Members of bands/groups with instruments, vocal type, and tenure FK: artist_id refs artists(id) ON DELETE CASCADE
nfo_snapshots Historical NFO file versions for undo/restore FK: artist_id refs artists(id) ON DELETE CASCADE

Platform and configuration tables:

Table Purpose
platform_profiles Image naming conventions per platform (Kodi, Emby, Jellyfin, Plex, Custom) with JSON-encoded filename arrays
connections External platform connections (Emby, Jellyfin, Lidarr) with encrypted API keys and feature flags
libraries Music libraries linked to connections or manual filesystem paths

User and access control tables:

Table Purpose
users User accounts with password hashes and roles
sessions User session tokens with expiry
api_tokens API key tokens with scopes, hashes, and revocation

Rules and quality tables:

Table Purpose
rules Metadata quality rules with categories, config, and automation mode
rule_violations Detected violations per rule/artist with severity and fix status
health_history Snapshots of overall metadata health score over time

Jobs, settings, and webhooks:

Table Purpose
bulk_jobs Batch operation tracking (type, mode, status, counts)
bulk_job_items Individual items within bulk jobs
scraper_config Scraper settings per scope (global or artist-specific)
settings Global key-value settings
webhooks Webhook endpoints for event notifications

Repository Pattern

Data access uses repository interfaces defined in internal/artist/repository.go. Each interface has a SQLite implementation in the same package.

Interface Responsibility Implementation
Repository Core artist CRUD, search, field updates sqlite_artist.go
ProviderIDRepository Provider identity lookups, batch retrieval, fetch timestamps sqlite_provider.go
ImageRepository Per-slot image metadata (exists, low_res, dimensions, phash) sqlite_image.go
MemberRepository Band member CRUD and upsert sqlite_member.go
AliasRepository Alias management, duplicate detection (MBID and alias-based) sqlite_alias.go
PlatformIDRepository Emby/Jellyfin/Lidarr platform ID mappings sqlite_platform.go

The artist.Service struct aggregates all repositories and provides the public API. Two constructors support both production use (NewService(db), which creates SQLite repos internally) and testing (NewServiceWithRepos(...), which accepts injected mocks).

Shared Database Helpers

internal/dbutil/ provides reusable type conversion functions for SQLite:

  • BoolToInt / IntToBool -- SQLite has no native boolean; values are stored as 0/1
  • ParseTime -- Parses multiple timestamp formats (RFC3339, datetime, date-only)
  • FormatNullableTime -- Converts *time.Time to a nullable SQL value
  • NullableString -- Converts empty strings to nil for nullable TEXT columns
  • NilableTime -- Converts *time.Time to *string (for JSON omitempty)

NFO System

NFO files are XML documents containing artist metadata in a format compatible with Emby, Jellyfin, and Kodi.

Key files:

  • internal/nfo/model.go -- ArtistNFO struct (name, MBIDs, genres, biography, image references, etc.)
  • internal/nfo/parser.go -- XML unmarshaling with HTML entity handling
  • internal/nfo/writer.go -- XML marshaling with round-trip fidelity
  • internal/nfo/snapshot.go -- Snapshot service (saves NFO state for undo/restore)
  • internal/nfo/conflict.go -- Last-modified timestamp check before writing
  • internal/nfo/diff.go -- Visual diff generation for the UI

Round-trip fidelity: Unknown XML elements are preserved via RawElement fields. If the NFO contains elements Stillwater does not recognize, they survive a parse-write cycle unchanged.

Conflict detection: Before writing an NFO file, the system checks the file last-modified timestamp. If the file was modified externally since it was last read, the write is blocked and the user is warned. This prevents overwriting changes made by other tools (Lidarr, manual edits).

Snapshots: The snapshot service saves the current NFO content before modifications, enabling undo and restore operations.

Event Bus

The event bus provides channel-based pub/sub for decoupling core operations from side effects.

flowchart LR
    P1[Scanner] -->|ScanCompleted| B[Event Bus]
    P2[Rule Pipeline] -->|RuleViolation| B
    P3[Bulk Executor] -->|BulkCompleted| B
    P4[Lidarr Webhook] -->|LidarrArtistAdd| B
    P5[FS Watcher] -->|FSDirCreated| B

    B -->|dispatch| S1[Webhook Dispatcher]
    B -->|dispatch| S2[SSE Hub]
    B -->|dispatch| S3[Scanner re-trigger]
Loading

Implementation (internal/event/bus.go):

Component Detail
Channel Buffered (capacity 256)
Subscribe(type, handler) Register a handler for an event type
Publish(event) Non-blocking send; drops with a warning log if the buffer is full
Start() Drains the channel, dispatches to subscribers (runs in a goroutine)
Stop() Signals shutdown, drains remaining events

Event types: ArtistNew, MetadataFixed, ReviewNeeded, RuleViolation, BulkCompleted, ScanCompleted, LidarrArtistAdd, LidarrDownload, FSDirCreated, FSDirRemoved

Dispatch behavior:

  • Handlers run synchronously per subscriber (one at a time per event)
  • Panics in handlers are caught and logged; other handlers still execute
  • Publishers never block on subscriber processing (the channel decouples them)

Image Processing

Key files:

  • internal/image/processor.go -- Format detection, dimension probing, resize, resolution checks
  • internal/image/save.go -- Atomic writes via internal/filesystem/
  • internal/image/naming.go -- Platform-specific filename conventions
  • internal/image/fanart.go -- Multi-fanart support (fanart.jpg, fanart-1.jpg, fanart-2.jpg, ...)
  • internal/image/cleanup.go -- Extraneous image removal

Format policy:

  • JPG and PNG only (no WebP, GIF, or other formats)
  • Logos are always PNG (preserves alpha transparency)
  • When saving a new image, existing files of the same type in other formats are deleted (e.g., saving folder.jpg deletes folder.png)

Resolution thresholds (used by rule checkers):

Type Minimum
Thumbnail 500x500
Fanart 960x540
Logo 400x155
Banner 758x140

Atomic file writes (internal/filesystem/atomic.go):

  1. Write data to <target>.tmp
  2. If target exists, rename to <target>.bak
  3. Rename .tmp to target
  4. Delete .bak
  5. If rename fails (cross-device), fall back to copy + delete + fsync

UI Layer

Templates:

  • Templ generates type-safe Go code from .templ files
  • Source templates in web/templates/ and web/components/
  • Generated _templ.go files are committed alongside source templates
  • Regenerate after changes: templ generate

HTMX patterns:

  • Dynamic page updates without full page reloads
  • Form submissions via HTMX attributes (hx-post, hx-swap, etc.)
  • Server returns HTML fragments that HTMX swaps into the DOM

CSS:

  • Tailwind CSS v4 via standalone CLI (no Node.js dependency)
  • Input: web/static/css/input.css
  • Output: web/static/css/styles.css
  • Build: tailwindcss -i web/static/css/input.css -o web/static/css/styles.css --minify

Vendored JS (in web/static/):

  • HTMX -- dynamic page updates
  • Cropper.js -- image cropping UI
  • Chart.js -- dashboard charts
  • Sortable.js -- drag-and-drop list reordering

Cache busting:

  • StaticAssets hashes each static file at startup using SHA-256
  • Templates receive cache-busted URLs (e.g., /static/css/styles.css?v=a1b2c3d4)
  • Matching hash triggers Cache-Control: public, max-age=31536000, immutable
  • New deploys produce new hashes; browsers fetch fresh copies automatically

Clone this wiki locally