Skip to content

v1.0.0: Somnia - Self-Hosted OpenRouter Batch Processing Dashboard

Latest
Latest

Choose a tag to compare

View all tags
@HoldenStill HoldenStill released this 12 Oct 05:27
· 1 commit to main since this release
v1.0.0
d9855eb

Somnia v1.0.0: Self-Hosted OpenRouter Batch Processing Dashboard

Somnia is an open-source, self-hosted web application tailored for running high-throughput, 50% discounted batch LLM inference jobs via OpenRouter. Packaged with zero external dependencies for Docker and TrueNAS Scale homelabs.

🚀 Highlights & New Features

  • OpenRouter Native Batch Execution: Full support for OpenRouter's inline JSON array batch architecture (POST /api/v1/batches with /v1/chat/completions) offering automated 50% discount routing.
  • Embedded Zero-Dependency Poller: Integrated AsyncIOScheduler background worker running inside FastAPI that polls active batch statuses on 5-minute intervals with manual refresh triggers.
  • Persistent SQLite Architecture: Non-blocking SQLite storage via aiosqlite with Write-Ahead Logging (WAL mode) preserving batches, individual item statuses, and user settings across container updates in /app/data/somnia.db.
  • System Prompt & JSON Structured Outputs: Comprehensive parameter configuration including system prompt overrides, temperature control, reasoning effort, and JSON mode enforcement.
  • Webhook Alert Dispatcher: Native outgoing webhook notifications for Discord, NTFY, and Gotify upon batch completion or failure.
  • Failed Row Isolation & 1-Click Retry: Detect errored tasks and spin off a retry batch with only the failed prompts via POST /api/batches/{id}/retry-failed.
  • Multi-Format Dataset Export: Export completed batch responses as raw CSV, structured JSON, or compiled Markdown (.md) reports.

📐 STEM & Engineering Math

  • Integrated Markdown & KaTeX Engine: Rich Markdown formatting supporting GitHub Flavored Markdown (GFM) tables, syntax-highlighted code blocks with 1-click clipboard copy, and LaTeX math typesetting with KaTeX.
  • Sample Electrical Engineering Benchmark: Included example_circuits.csv dataset featuring real-world analog circuit specs (Sallen-Key filters, series RLC band-pass, high-frequency differentiators, Schmitt triggers) for batch analysis and Python simulation scripts.

💰 Inference Cost & OpenRouter Accounting

  • Batch Endpoint Validation: Strict validation requiring models to have active :batch endpoints, automatically appending :batch slugs and preventing 400 validation failures.
  • Pre-Flight Cost Estimator: Immediate pre-submission token and cost calculation factoring in 50% batch savings against live OpenRouter pricing.
  • Persistent Spend & Token Accounting: Detailed token tracking (prompt, completion, total) and monetary cost accounting surfaced in a 5-card Stats Summary Bar, top header badges, and per-task cost pills.

🧭 UI/UX Ergonomics

  • Dual-Mode Item Navigator: Toggle between Focused Item View (scrollable row tab bar, row-picker jump dropdown, and ← / → arrow key navigation) and All Rows (Feed) view to eliminate vertical scroll fatigue.
  • Collapsible Slim Sidebar & Resizable Divider: Gemini-style collapsible sidebar shrinking to a 64px icon rail, with a smooth draggable divider allowing sidebar width adjustment between 240px and 500px (persisted in localStorage).
  • Row 1 Prompt Preview: Real-time prompt interpolation preview directly in the batch submission dialog before launching jobs.
  • Unread Notification Dots: Pulsing teal notification badges identifying uninspected batches until acknowledged.

🐳 Docker & Self-Hosted Infrastructure

  • Container Registry Distribution: Automated multi-architecture (linux/amd64, linux/arm64) Docker container publishing to Docker Hub (holdenstill/somnia:latest).
  • TrueNAS Scale & Homelab Optimization: Out-of-the-box support for PUID/PGID (1000:1000) non-root user permissions, persistent volume mounts (/app/data), and complete TrueNAS Scale deployment documentation (TRUENAS_SETUP.md).

🛠️ Bug Fixes & Hardening

  • Model Slug Normalization: Automated fallback mapping and validation preventing non-batch model rejections.
  • Discount Arithmetic Correction: Resolved pre-flight double-discount calculations.
  • Comprehensive Test Coverage: Automated test suite covering API endpoints, CSV templating, database CRUD, and polling logic with 100% test pass rate.

📦 Downloadable Release Assets

  • docker-compose.yml: Standard self-hosted Docker deployment manifest.
  • example_circuits.csv: Sample electrical engineering circuit design dataset.
  • .env.example: Configuration template for API keys and container environment variables.

🤖 Attribution

Crafted autonomously by Gemini Antigravity (Google DeepMind) • Conceived and directed by Holden.

Assets 5
Loading