pi-posthog

PostHog LLM analytics for the pi coding agent. Every agent run is captured as a full trace in PostHog AI observability — model calls with tokens, cost, and latency; tool executions; and the conversation content — so you can watch, debug, and measure your agent sessions like any other LLM product.

pi	PostHog
session	$ai_session_id (groups traces into a conversation)
prompt → agent run	one trace, closed with an $ai_trace event
each LLM call	$ai_generation — model, tokens, cost, latency, TTFT, content
each tool execution	$ai_span (type tool), nested under the generation that called it
context compaction	$ai_span (type compaction)

Costs computed by pi's model registry are passed through ($ai_*_cost_usd); when pi has no pricing for a model, PostHog computes cost from $ai_model automatically.

Install

pi install npm:pi-posthog
# or from a checkout
pi install /path/to/pi-posthog

Or drop extensions/posthog.ts into ~/.pi/agent/extensions/ — it's a single file with zero runtime dependencies.

Requires pi ≥ 0.80 and a PostHog project (Cloud US/EU or self-hosted).

Configure

Set your PostHog project API key (phc_…, Settings → Project → API key):

export PI_POSTHOG_API_KEY=phc_your_key
# EU cloud or self-hosted:
export PI_POSTHOG_HOST=https://eu.i.posthog.com

Then just use pi. Traces appear under AI observability → Traces in PostHog. Run /posthog test inside pi to verify connectivity.

All settings, via env vars or JSON config (~/.pi/agent/posthog.json for the user, .pi/posthog.json per project — project wins, env vars win over both):

JSON key	Env var	Default	Meaning
apiKey	PI_POSTHOG_API_KEY / POSTHOG_API_KEY	—	Project API key. Without it the extension stays inert.
host	PI_POSTHOG_HOST / POSTHOG_HOST	https://us.i.posthog.com	Ingestion host.
distinctId	PI_POSTHOG_DISTINCT_ID	pi:<user>@<host>	Person the events belong to.
captureContent	PI_POSTHOG_CAPTURE_CONTENT	true	Capture prompts, completions, and tool input/output.
captureThinking	—	false	Include model thinking blocks in captured output.
captureSystemPrompt	—	false	Reserved: include the system prompt in $ai_input.
captureTools	—	true	Attach active tool names/descriptions as $ai_tools.
statusBar	—	true	Show a posthog <n> event counter in pi's footer.
enabled	PI_POSTHOG_DISABLED=1 to disable	true	Master switch.
debug	PI_POSTHOG_DEBUG	false	Log activity to ~/.pi/agent/posthog-debug.log.
flushAt / flushIntervalMs / maxQueueSize	—	20 / 3000 / 1000	Batching knobs.

Example ~/.pi/agent/posthog.json:

{
	"apiKey": "phc_your_key",
	"host": "https://us.i.posthog.com",
	"captureContent": true
}

Privacy

• captureContent: false (or PI_POSTHOG_CAPTURE_CONTENT=0) sends metadata only — model, tokens, cost, latency, tool names — and no prompt, completion, tool arguments/output, or prompt-derived trace names.
• Thinking blocks are redacted by default.
• Content is truncated client-side (8k chars per message, 40 messages per input, 20k per tool state) before leaving your machine.
• Events are batched and sent asynchronously; failures never block or break the agent, and the queue is flushed on session shutdown.

/posthog command

Inside pi:

• /posthog — status: host, distinct id, sent/queued/dropped counters, last error
• /posthog test — send a connectivity test event and flush
• /posthog flush — flush the queue now
• /posthog on / /posthog off — toggle capture for this session

What a trace looks like

Each pi prompt becomes a trace named after the first line of your prompt, with generations (turn 0, turn 1, …) at the top level and tool spans nested under the generation that invoked them:

▾ Fix the failing auth test            $ai_trace      12.4s  $0.0214
  ▾ turn 0                             $ai_generation  3.1s   1.2k → 86 tok
      bash                             $ai_span        0.4s
  ▾ turn 1                             $ai_generation  2.8s   1.4k → 210 tok
      edit                             $ai_span        0.0s
  ▾ turn 2                             $ai_generation  4.2s   1.7k → 340 tok

Extra pi-specific properties are attached for filtering: pi_session_id, pi_turn_index, pi_tool_name, pi_turns, pi_generations, pi_tool_calls, pi_model, pi_api.

Development

npm install
npm run typecheck   # tsc --noEmit
npm test            # simulated pi session against a mock PostHog server (Node ≥ 23.6)

The test drives the real extension through a full session (two LLM calls, a tool call, a redacted run) and asserts on the captured $ai_* payloads — no network or API key needed.

To test live against your project:

PI_POSTHOG_API_KEY=phc_… pi -e ./extensions/posthog.ts -p "say hi"

License

MIT