GAFFER

GAFFER is a release-night control room for streaming operations. It turns playback telemetry and infrastructure state into one traceable incident verdict, then keeps every external write behind a named human approval.

Built for Agentic Cinema: The Blockbuster Hackathon, ClickHouse track.

Live project: gaffer-648281515762.asia-south1.run.app

Source: github.com/himanshu748/gaffer

What is real

The demo uses synthetic playback telemetry so no customer or personal data is involved. The integrations and runtime calls are real:

Component	Runtime role
ClickHouse Cloud	Holds one million synthetic playback events, cohort baselines and the append-only agent ledger
Official mcp-clickhouse	Performs every telemetry read used by the crew
Grafana Cloud with official mcp-grafana	Reads infrastructure context and performs the only implemented external write, a human-approved annotation
Gemini 2.5 Flash through Google ADK	Produces a schema-validated cross-source verdict. It is the only AI model in the submission path
Vertex AI Agent Engine Runtime	Hosts the submission-path ADK Director as a managed Google Cloud agent
Cloud Run	Serves the public control room, API and WebSocket stream

Each live API cycle re-reads the latest seeded release window through the official ClickHouse MCP. Anchoring the analysis to the newest synthetic event keeps the public drill replayable, while preflight separately fails stale telemetry before a recording.

The five-step crew

WATCH -> LIGHT -> CUT -> ROLL -> PRINT

1. WATCH: the Cinematographer queries ClickHouse for regional playback deviation.
2. LIGHT: the Gaffer reads Grafana for recent changes and firing infrastructure state.
3. CUT: the managed ADK Director fuses ClickHouse and Grafana evidence into one structured verdict.
4. ROLL: policy code proposes actions. It never lets the model dispatch them.
5. PRINT: every step and approval is written to the ClickHouse audit ledger.

The order is code, not a model decision. Gemini interprets evidence, it does not choose which tools to call or bypass approval policy.

Honest impact model

Viewer counts, failure rates and affected subscribers come from ClickHouse queries. The financial number is a planning scenario, not observed studio economics. The control room prints its configurable demo assumptions beside the result:

• 3.4% potential churn per attributable failed premiere start
• EUR 155.88 annual demo ARPU

A production deployment must replace both with an approved finance model.

Run locally

Requirements: Python 3.12, a ClickHouse Cloud connection, a Grafana service-account token and either a Gemini API key for local development or a deployed Agent Engine resource.

cd backend
python3.12 -m venv ../.venv
../.venv/bin/pip install -r requirements.txt
cp .env.example .env

../.venv/bin/python scripts/bootstrap.py
../.venv/bin/python scripts/premiere_sim.py --clean --rows 1000000 --anchor 47
../.venv/bin/python scripts/preflight.py
../.venv/bin/python scripts/run_cycle.py
../.venv/bin/uvicorn gaffer.api.main:app --port 8080

Open http://127.0.0.1:8080. Live mode is the default. Add ?mode=demo only for the permanently labeled synthetic replay. Live mode never falls back to fixture data.

The official Grafana MCP binary is included in the container build. For local development, place a compatible mcp-grafana binary on PATH or in backend/bin/.

Verification

cd backend
../.venv/bin/pytest -q
../.venv/bin/python -m compileall -q gaffer scripts
cd ..
node --check control-room/app.js
node --check control-room/data.js

scripts/preflight.py checks the submission-path integrations. Its Gemini check uses the configured Director, so production preflight verifies Agent Engine rather than a separate API-key path. It also requires the newest ClickHouse event to be within five minutes by default.

Security and cost controls

• .env is ignored, excluded from both container contexts and kept local by default.
• Cloud secrets are separate Secret Manager values visible only to the runtime identity.
• The Gemini API key is not needed in Cloud Run or Agent Engine.
• The public approval route requires a bearer token plus a named operator.
• Public live sessions end after three minutes and are capped below Cloud Run request capacity.
• Unimplemented action dispatchers remain approved, never falsely marked dispatched.
• Agent Engine and Cloud Run both scale to zero with one maximum instance.
• Cloud Run keeps CPU available while an instance is active so the background crew does not freeze between requests.
• The Google Cloud billing budget is project-scoped, excludes credits and alerts at 50%, 75%, 90% and 100% of the INR 4,000 target. Google budgets alert, they do not stop spend.

Repository map

control-room/                 live-first judge interface and explicit demo replay
backend/gaffer/agents/        crew orchestration and Google ADK Director
backend/gaffer/mcp/           ClickHouse and Grafana submission-path adapters
backend/gaffer/analysis/      viewer and modeled financial exposure
backend/gaffer/policy/        human approval gates
backend/gaffer/api/           FastAPI, WebSocket and static hosting
backend/scripts/              bootstrap, synthetic seed, preflight and deployment
docs/hackathon-build/         accepted scope, PRD, spec and verification checklist

License

Apache License 2.0. See LICENSE.