Architecture

Architecture

Authoritative walkthroughs: backend/README.md (every flow, traced through the code) and the design dossier, especially 02-system-architecture and 09-database-schema.

Write path

Deliberately mirrors Langfuse's proven design, reimplemented in Python, with agent semantics (agent_id, conversation_id, turn, step, tool_call_id, env) promoted to first-class indexed columns:

SDK / OTLP → POST /v1/traces → S3 blob (durable FIRST) → Redis / Celery
  → worker: otel/ mapping → registry upsert → ClickHouse events
  → evaluate_run_task (countdown=4s, debounces late spans) → scores + structural clustering

One adaptation: ClickHouse server-side async_insert instead of an in-process write buffer, because Celery tasks don't share memory (why).

Five stores

Store	Holds
ClickHouse	events (one row per span) + scores, both ReplacingMergeTree — re-ingest / re-eval converge instead of duplicating
Postgres + pgvector	The registry: projects, keys, agents, cases, gates, clusters, evaluators, users, orgs, monitors, annotations, assistant chats; failure embeddings
S3 / MinIO	Raw OTLP bodies (source of truth) + regression fixture bundles + assistant attachments
Redis	Celery broker
OpenRouter (per workspace)	Judges, failure-intel agents, rolling summary, meta-analysis — on the customer's own key

Code layout

A uv workspace (backend, workers, sdk) + a pnpm Next.js app (frontend) + a Nextra docs site (docs).

backend/tracely/ is one package with strict layering:

Layer	Rule
domain/	Pure logic, no I/O — stats, verdict policy, contracts, trajectories, templates
infrastructure/	Every adapter: clickhouse/, db/, blob/, queue/, llm/, notifications/, registry/
services/	Use-case orchestrators (IngestionService, EvaluationService, GateService, …)
api/, workers/	Thin. Routers shape HTTP; Celery tasks dispatch into a service

workers/ is a deployable shim importing tracely.workers.tasks. sdk/ is the instrumentation SDK and the tracely simulate / gate / replay CLI.

Agents reach the product through the API, never the DB

Both the MCP server (/mcp) and the in-app assistant's tools call api/internal_client.api_call, which re-issues the request against our own ASGI app with the caller's own credentials. Scoping therefore happens in exactly one place (get_project_id), and the routers' own 4xx bodies are what the agent reads to correct itself. The assistant's model runs on Tracely's key; its tools run as the user.

Frontend

Next.js 15 App Router + Tailwind. Server Components call app/lib/api.ts directly; Client Components fetch a thin proxy under app/api/* that re-issues with the Bearer key server-side — TRACELY_KEY / TRACELY_API never reach the browser. Live evaluator scores arrive over SSE into a per-cell store so a frame re-renders one cell, not the grid. Details: frontend/README.md.

Auth

AUTH_MODE = dev (open; prod refuses to boot) · local (email/password + JWT, SESSION_SECRET) · clerk. Organizations sit above workspaces; a workspace is reachable exactly when the user is a member of the org owning it. Ingest keys are machine credentials bound to one project and carry no role — destructive endpoints require a signed-in human.