omen

Divines disaster, crafts the cure.

omen is an AI agent that load-tests a code change, finds the regression in live SigNoz traces, and writes the fix, before it reaches production. Point it at a git diff or a plain-language intent. It drives real k6 load against the affected endpoints, correlates the result with server-side telemetry through the official SigNoz MCP Server, names the root cause, and hands back a validated remediation diff. For engineering and SRE teams who need to catch load-induced regressions before a 2am page.

omen runs as an audited Burr state machine over MCP, served by Theodosia: the driving agent sees one tool, illegal steps are refused, and every step and refusal lands on a hash-chained ledger. Named for Kassandra, who foresaw what others would not believe; the workflow is themed as a tarot draw, one Major Arcana card per phase (omen arcana lays out the spread).

Built for Agents of SigNoz (Track 01 · AI & Agent Observability). Setup: docs/SIGNOZ_SETUP.md. Writeup: Dev.to blog.

Screenshots

omen render: the full state machine, legal edges only	omen arcana: a card per phase
omen doctor --runtime: graph + governance checks	a full run: k6 + SigNoz correlated, every step on the ledger

What it does

• Diff or intent driven. Reads the changed endpoints from a git diff, or scores an OpenAPI spec against a plain-language intent.
• Real load. Generates and runs an actual k6 test through the Grafana k6 MCP server, on top of a deterministic scaffold so a run never fails for lack of a model.
• Server-side truth from SigNoz. Correlates the run with trace aggregates through the official SigNoz MCP Server (signoz_aggregate_traces, signoz_search_traces) over the test window. Catches latency degradations that produce zero errors and would slip past any threshold alert.
• Root cause and a fix. Writes a cited analysis and a validated remediation diff that applies cleanly, screened by an independent auditor model before the verdict is sealed.
• Audited by construction. A governed state machine refuses illegal steps; every step and refusal is on a hash-chained ledger that omen verify checks.
• Hands-free guard. omen watch runs the whole workflow when a commit changes an endpoint, catching the regression at commit time.
• Model-agnostic. The same loop (drive, write, audit) runs on Gemini, a local open model, or a frontier API, unchanged.
• Self-observable. Publishes its own state-machine walk as OTLP traces to SigNoz, so the agent is visible in the system it observes.

Install

Prerequisites: Python 3.12+, uv, Docker (for SigNoz Foundry), and k6 2.0+ on your PATH.

uv sync

After sync, run CLI commands as uv run omen … (or activate .venv so bare omen works). omen is not installed globally by uv sync alone.

omen drives k6 through the Grafana k6 MCP server and reads SigNoz through the official SigNoz MCP server. Full setup: docs/SIGNOZ_SETUP.md.

foundryctl cast -f casting.yaml   # SigNoz UI :8080, OTLP :4317, MCP :8000/mcp
cp .env.example .env              # PowerShell: Copy-Item .env.example .env
# Edit .env: set OMEN_SIGNOZ_API_KEY and GEMINI_API_KEY (key alone on each line — no trailing comments)

Install k6 2.0+, then warm the MCP extension cache:

# macOS (Homebrew):
brew install k6
# Windows: install from https://grafana.com/docs/k6/latest/set-up/install-k6/ (or scoop/choco), ensure k6 is on PATH
# Linux: see k6 docs, or use Docker only if host networking reaches 127.0.0.1 (see below)

uv run omen warm-k6

.env.example defaults to local k6 (OMEN_K6_CMD=k6 x mcp). That is the supported path on Windows and macOS.

• Do not set OMEN_K6_DOCKER=1 on Windows — Docker --network host cannot reach demo apps on 127.0.0.1.
• On Linux only, you may set OMEN_K6_DOCKER=1 so the container uses host networking to hit local demos.
• Standalone binary alternative: OMEN_K6_CMD=mcp-k6.

Optional: install extra OpenTelemetry instrumentors. uv sync already pulls FastAPI + OTLP packages used by the demos; bootstrap needs pip in the venv:

uv run python -m ensurepip --upgrade   # uv venvs often omit pip
uv run opentelemetry-bootstrap --action=install

The SigNoz step is optional: without OMEN_SIGNOZ_API_KEY, omen skips correlation and runs k6-only. Model backend: see Configuration.

Quickstart

The fastest first success needs no SigNoz, k6, or model. It runs the whole state machine against fakes:

uv run pytest                 # full workflow against FakeUpstream + fake model
uv run omen render            # print the state machine
uv run omen arcana            # the tarot spread, one card per phase
uv run omen doctor --runtime  # graph + runtime tool shape

For a full live run against a bundled demo app (starts the app under opentelemetry-instrument, drives real k6, queries live SigNoz, writes the grounded analysis):

uv run python scripts/verify_scenario.py petclinic   # or storefront | feed | gateway | orders
# ports: petclinic 8400 · storefront 8401 · feed 8402 · gateway 8403 · orders 8404

For a recorded demo (terminal cast → GIF; Linux/macOS with bash, asciinema, and related tools — not a Windows PowerShell path):

./scripts/record_demo_cast.sh   # asciinema + agg → docs/assets/omen-run.gif
./scripts/record_demo.sh        # X11 screen capture → docs/assets/omen-run.mp4

Usage

Inspect and serve the workflow:

uv run omen doctor --runtime     # validate the graph and runtime tool shape
uv run omen render               # print the state machine
uv run omen serve                # mount as an MCP server over stdio (both upstreams wired in)

Drive it locally with omen pilot. The driver depends on OMEN_LLM:

• ollama — a local open model (e.g. Granite) reads reachable actions and calls step each turn.
• gemini / anthropic / claude_agent — the same governed MCP FSM is walked; that backend authors, correlates, analyzes, and screens.

Start a demo app first (or use verify_scenario.py, which starts it for you). Runnable demos live under examples/petclinic (and storefront / feed / gateway / orders), not examples/petstore (OpenAPI-only fixture for unit tests).

# terminal 1 — instrumented demo
uv run opentelemetry-instrument python examples/petclinic/app.py serve

# terminal 2 — intent mode (Gemini when OMEN_LLM=gemini)
uv run omen pilot \
  --intent "load test recording a new visit" \
  --repo-path examples/petclinic \
  --target-base-url http://localhost:8400 \
  --signoz-service petclinic

# or diff mode against a repo with two commits that change an endpoint:
# uv run omen pilot --repo-path /path/to/repo --ref HEAD~1 \
#   --target-base-url http://localhost:8400 --signoz-service petclinic

Run it in the background, triggered on diff detection. omen watch polls a repo's git HEAD and, when a new commit changes an HTTP endpoint, drives the whole workflow in diff mode against that change, then prints the verdict and a proposed fix and publishes the run to SigNoz over OTLP:

uv run omen watch --repo-path /path/to/repo \
  --target-base-url http://localhost:8400 --signoz-service petclinic
# one-shot for a post-commit hook or CI:
# uv run omen watch --once --repo-path . --target-base-url http://localhost:8400 --signoz-service petclinic

Or drive it from Claude Code (or any MCP client) by registering the server:

claude mcp add --scope=user --transport=stdio omen -- uv run omen serve

Then ask the agent to run the workflow with the step tool, for example: "Use the omen step tool. Load test recording a new visit against http://localhost:8400; the app and OpenAPI are under examples/petclinic; correlate with SigNoz service petclinic."

The entry inputs for select_mode:

• diff mode: {"repo_path": "/path/to/repo", "ref": "HEAD~1", "target_base_url": "http://localhost:8400", "signoz_service": "petclinic"}
• intent mode: {"repo_path": "examples/petclinic", "intent": "load test recording a new visit", "target_base_url": "http://localhost:8400", "signoz_service": "petclinic"}

Review recorded runs (best after omen pilot / MCP step, which writes a full hash-chained ledger):

uv run omen sessions ls
uv run omen sessions show <app-id>
uv run omen logs <app-id> --refusals
uv run omen verify <app-id>        # confirm the ledger has not been tampered with

scripts/verify_scenario.py is the fastest live k6+SigNoz check; it uses Burr arun and may not show the same multi-step ledger listing as omen pilot. Use pilot when you need omen verify for the demo.

Configuration

Variable	Default	Purpose
OMEN_LLM	gemini	model backend: gemini (Google GenAI SDK), ollama (local), claude_agent, or anthropic
OMEN_MODEL	gemini-3.1-flash-lite	model id for the selected backend
GEMINI_API_KEY	unset	Gemini API key (for OMEN_LLM=gemini)
OLLAMA_HOST	http://localhost:11434	Ollama endpoint (for OMEN_LLM=ollama)
ANTHROPIC_API_KEY	unset	Claude API key (for OMEN_LLM=anthropic)
OMEN_GUARDIAN	unset	set to enable the independent groundedness screen
OMEN_K6_CMD	k6 x mcp	command line for the local k6 MCP server (or mcp-k6)
OMEN_K6_DOCKER	unset	if set, run the k6 MCP via Docker (Linux host-network only; avoid on Windows)
OMEN_K6_IMAGE	grafana/mcp-k6:latest	Docker image when OMEN_K6_DOCKER is set
OMEN_SIGNOZ_URL	http://localhost:8080	SigNoz API base URL
OMEN_SIGNOZ_API_KEY	unset	service-account API key (SIGNOZ-API-KEY header)
OMEN_SIGNOZ_FLUSH_SEC	unset	seconds to wait after k6 before SigNoz correlate
OTEL_EXPORTER_OTLP_ENDPOINT	unset	OTLP endpoint for omen + demo app traces (e.g. http://localhost:4317)
OTEL_EXPORTER_OTLP_PROTOCOL	grpc	OTLP transport
THEODOSIA_HOME	~/.omen	ledger / session store
THEODOSIA_LEDGER_KEY	unset	optional HMAC key so omen verify reports HMAC-keyed integrity

omen / omen serve load these from a .env in the project root if present (see .env.example); real environment variables take precedence. Keep .env out of git (it is git-ignored) since keys are credentials. Use the filename .env — not .env.txt.

When OMEN_K6_DOCKER=1 (Linux), omen uses --network host so the container can reach demo apps on 127.0.0.1:<port>.

How it works

stateDiagram-v2
    [*] --> select_mode
    select_mode --> read_diff: diff mode
    select_mode --> parse_intent: intent mode
    read_diff --> extract_endpoints
    read_diff --> analyze: failed
    extract_endpoints --> doc_lookup
    parse_intent --> doc_lookup
    parse_intent --> analyze: failed
    doc_lookup --> scaffold
    scaffold --> generate_script
    scaffold --> analyze: no endpoints
    generate_script --> validate_script
    validate_script --> fix_script: needs fix
    fix_script --> validate_script
    validate_script --> run_test: valid / scaffold
    validate_script --> analyze: gave up
    run_test --> signoz_preflight: signoz enabled
    run_test --> analyze: k6-only / failed
    signoz_preflight --> correlate
    correlate --> detect_anomalies
    detect_anomalies --> analyze
    analyze --> screen
    screen --> report
    report --> [*]

Loading

(generated by omen render --mermaid)

• doc_lookup — consults the k6 MCP documentation tools and records version-grounded citations. Non-blocking; generation proceeds if the docs are unavailable.
• scaffold — composes a deterministic, self-contained k6 baseline from the OpenAPI schema (per-endpoint requests with sample bodies, baked base URL, load options). No model. This scaffold is the known-good fallback.
• generate_script — the model authors the final script on top of the scaffold, guided by k6's own generate_script MCP prompt and best_practices resource.
• validate_script — gates the script at k6.validate_script. Failures route to fix_script, which repairs the script from the real k6 error (stderr + the server's structured issues and suggestions) and loops back. Bounded by MAX_FIX_ATTEMPTS; gives up cleanly to the scaffold so an unvalidated script never reaches run_test.
• run_test — executes the validated script via k6.run_script and records the wall-clock test window.
• signoz_preflight — checks the target service exists in SigNoz (signoz_list_services) before correlating. Non-blocking.
• correlate — SigNoz MCP trace aggregates and searches synthesize client-vs-server findings: which route degraded and why.
• detect_anomalies — time-series p95 buckets from signoz_aggregate_traces flag latency breaches over the test window. Non-blocking.
• analyze — the writer model produces a grounded analysis (root cause, evidence with source citations, recommendation) and, in diff mode, a proposed remediation diff that applies cleanly. Both fall back to deterministic text when no model is available.
• screen — an independent auditor model checks every claim in the analysis against the cited telemetry. Non-blocking.
• report — assembles the combined client-plus-server verdict, seals it to the ledger, and publishes the run and the agent's state-machine walk to SigNoz over OTLP.

The Major Arcana

Each phase is a card the agent turns. Run uv run omen arcana for the full spread.

Card	Phase	Omen
The Fool (0)	select_mode	the querent sets out: diff or intent
The High Priestess (II)	read_diff	hidden knowledge read from the diff
The Emperor (IV)	extract_endpoints	order from change: the routes are named
The Empress (III)	parse_intent	intuition reads the intent into endpoints
The Hierophant (V)	doc_lookup	doctrine consulted: the k6 docs ground the rite
The Chariot (VII)	scaffold	the vehicle is assembled from the spec: a runnable scaffold takes shape
The Magician (I)	generate_script	as above, so below: the agent authors the script atop the scaffold
Justice (XI)	validate_script	the script is weighed; the unworthy is turned back
Temperance (XIV)	fix_script	the flawed draft is tempered against k6's judgment until it holds
The Tower (XVI)	run_test	load strikes the structure; what breaks is revealed
The Hermit (IX)	signoz_preflight	a lantern into SigNoz before the reading
The Lovers (VI)	correlate	client and server joined over one window
The Star (XVII)	detect_anomalies	the latency band is cast; where load breaches it is revealed
The Sun (XIX)	analyze	the reading is made plain: cause, evidence, and the cure laid bare
The Hanged Man (XII)	screen	seen again through another's eyes: the reading is judged grounded, or not
Judgement (XX)	report	the verdict is spoken and sealed to the ledger
The World (XXI)	the ledger	the cycle closes: an immutable, hash-chained record
The Devil (XV)	a refusal	you are bound: only the legal moves are permitted

Demo scenarios

examples/ ships five runnable target apps, each a healthy baseline plus one "new" endpoint with a distinct load-induced failure. Each runs under opentelemetry-instrument and exports traces to SigNoz. (examples/petstore is an OpenAPI-only fixture for offline tests — it has no app.py.)

App	Port	New endpoint	Failure signature	What it exercises
petclinic	8400	POST /api/visits	5xx, constant, database is locked	correlation isolates the root-cause error
storefront	8401	POST /api/checkout	latency, 0 errors (N+1 over a shared connection)	server-side latency invisible to the client error rate
feed	8402	POST /api/events	latency rising over the run (unbounded recompute)	detect_anomalies catches the trend
gateway	8403	GET /api/quote	4xx 429 throttling (too-tight rate limit)	client-vs-server error split: "throttled, not broken"
orders	8404	POST /api/order	latency + 504 mix (downstream cascade)	dependency root cause, resilience recommendation

uv run python scripts/verify_scenario.py petclinic   # or storefront | feed | gateway | orders
uv run python scripts/setup_dashboard.py             # SigNoz dashboard (official REST API)

Development

uv run ruff format . && uv run ruff check .
uv run pytest

The tests use Theodosia's FakeUpstream for both MCP servers and a fake LLM, so they run offline with no k6, SigNoz, Gemini, or network.

Windows

• CLI: use uv run omen … (or .venv\Scripts\Activate.ps1 then omen).
• Terminal encoding: omen and scripts/capture_run.py reconfigure stdout to UTF-8 so the tarot sigil (🂠) does not crash default cp1252 consoles. Windows Terminal is recommended; alternatively set PYTHONIOENCODING=utf-8.
• k6: install a local k6 2.0+ binary; set OMEN_K6_CMD=k6 x mcp. Do not set OMEN_K6_DOCKER=1 — Docker --network host cannot reach demo apps on 127.0.0.1 on Windows.
• Config: Copy-Item .env.example .env (filename must be .env, not .env.txt).
• OTel bootstrap: if opentelemetry-bootstrap fails with No module named pip, run uv run python -m ensurepip --upgrade first, or skip it — uv sync already installs the FastAPI instrumentation demos need.
• Demo GIFs: scripts/record_demo*.sh need bash/WSL; for local rehearsal use uv run python scripts/verify_scenario.py petclinic or uv run omen pilot ….

License

Apache-2.0. omen builds on Theodosia (Apache-2.0), Burr (Apache-2.0), the official Grafana k6 and SigNoz MCP servers, and the Google GenAI SDK for Gemini.

Tarot icon by Eucalyp from the Noun Project, CC BY 3.0.