Subsystems

Subsystems

One line per Go package under internal/ (plus the CLI). Grouped by the plane they serve.

Data plane

Package	Does
gatherer	batches incoming events (goroutine + channel) and flushes to ClickHouse; quarantines bad rows
pipe	parses .pipe templates ({{Type()}}, {% if/for %}), composes node SQL, executes endpoints
sqlproxy	read-only ad-hoc SQL (/v0/sql), readonly=2 enforced
clickhouse	the ClickHouse client: DDL builders, queries, materialized views, backfill

Control plane

Package	Does
datasource	parses .datasource files (SCHEMA + ENGINE)
deploy	applies .datasource/.pipe to ClickHouse; migrations; branch targeting
branch	resolves the current git branch → workspace DB name
model	shared domain types — Datasource, Pipe, registry (the vocabulary everything shares)

Serving & ops

Package	Does
api	chi router, handlers, middleware — the HTTP surface
config	layered runtime config (env, flags, file)
auth / token	scoped bearer tokens (ADMIN / APPEND:<ds> / READ:<pipe>)
ratelimit	per-token rate limiting (in-memory default)
pipestats	records per-query stats to the pipe_stats table
metrics	Prometheus metrics middleware

CLI

cmd/tinyraven — the single binary. See CLI.