Skip to content

Arize-ai/coding-harness-tracing

Repository files navigation

Arize Coding Harness Tracing

Trace AI coding sessions to Arize AX or Phoenix with OpenInference spans. Each harness integration emits spans for prompts, tool calls, model responses, and session lifecycle events.

Supported Harnesses

Harness Integration Install command Name
Claude Code CLI / Agent SDK macOS / Linux · Windows claude
Claude Code CLI / Agent SDK Claude Plugin claude-code-tracing
OpenAI Codex CLI macOS / Linux · Windows codex
Cursor IDE / CLI macOS / Linux · Windows cursor
Cursor IDE / CLI Cursor Plugin cursor-tracing
GitHub Copilot (VS Code + CLI) macOS / Linux · Windows copilot
Gemini CLI macOS / Linux · Windows gemini
Kiro CLI macOS / Linux · Windows kiro
Opencode CLI macOS / Linux · Windows opencode
Oh My Pi (omp) macOS / Linux · Windows omp

Each install link opens the ready-to-paste command for your OS — copy it and run it in a terminal

Installing Claude Code tracing via the Claude marketplace? See Claude Code Tracing for the marketplace-specific flow — backend credentials must be set directly in ~/.claude/settings.json since the install wizard is skipped.

Installing Cursor tracing via the Cursor marketplace? The cursor-tracing plugin registers all hook events automatically; run the bundled manage-cursor-tracing skill once after install to write backend credentials to ~/.arize/harness/config.json. See Cursor IDE Tracing for the full flow.

Setup walkthrough

The installer involves a brief interactive setup. The steps below run in order:

1. Backend selection

Choose where spans should be sent:

  • 1) Phoenix — your own Phoenix instance.
  • 2) Arize AX — the hosted Arize platform.

2. Credentials

Prompts depend on the backend:

  • Phoenix:
    • endpoint (defaults to http://localhost:6006)
    • optional API key (leave blank for no auth)
  • Arize AX:
    • Arize API key
    • Space ID (found in Arize settings tab along with api keys)
    • OTLP endpoint (defaults to otlp.arize.com:443 — only override for hosted/dedicated instances).

If you've already configured another harness against the same backend, the installer offers a copy-from menu so you can reuse those credentials instead of re-entering them.

3. Project name

The project (in Arize/Phoenix) that spans for this harness are grouped under. Defaults to the harness name (e.g. claude-code, codex etc).

4. User ID (optional)

A free-form identifier attached to every span as user.id. Useful when multiple teammates share the same backend. Leave blank to skip.

5. Content logging

Three Y/n opt-outs that apply to all harnesses:

  • Log user prompts?
  • Log what tools were asked to do (commands, file paths, URLs)?
  • Log what tools returned (file contents, command output)?

You're only asked these the first time you install a harness — subsequent installs reuse the existing logging: block. You can edit them later in ~/.arize/harness/config.json.

Environment variables

Most settings live in .arize/harness/config.json, but a small set of env vars affect runtime behavior on every harness. The installers wire most of these for you; set them yourself when you want to override behavior for a single session or debug locally.

Variable Default Description
ARIZE_TRACE_ENABLED true Master toggle. Set to false to disable hooks without uninstalling.
ARIZE_VERBOSE false Enables [arize] ... log lines in ~/.arize/harness/logs/<harness>.log. Errors are always logged; verbose adds routine activity (hook fires, span emits, state transitions).
ARIZE_DRY_RUN false Build spans but skip the backend send. Useful for confirming hook wiring without writing data.
ARIZE_USER_ID Attached to every span as user.id. Mirrors the user_id field in config.json; env wins if both are set.
ARIZE_PROJECT_NAME per-harness Overrides harnesses.<name>.project_name from config.json for a single session. Arize backend only — ignored on the Phoenix backend (use PHOENIX_PROJECT there).
ARIZE_LOG_FILE per-harness Path the harness writes its log to. Adapters default to ~/.arize/harness/logs/<harness>.log.
ARIZE_TRACE_DEBUG false Dump raw hook payloads as JSON under ~/.arize/harness/state/<harness>/debug/. Codex hooks use this for span-tree inspection.
OTEL_RESOURCE_ATTRIBUTES Standard OTel attribute string (team=payments,environment=prod) added to every span. Overrides config.json attributes/harnesses.<name>.attributes on key collision; set per-harness by placing it in that harness's settings env block.

Backend overrides (set if you want env to take priority over config.json for a single run):

Variable Description
ARIZE_API_KEY, ARIZE_SPACE_ID, ARIZE_OTLP_ENDPOINT Arize AX credentials and endpoint.
PHOENIX_ENDPOINT, PHOENIX_API_KEY Phoenix endpoint and (optional) API key.
PHOENIX_PROJECT, PHOENIX_PROJECT_NAME Project override on the Phoenix backend (mirrors ARIZE_PROJECT_NAME for Arize). PHOENIX_PROJECT wins if both are set; both override harnesses.<name>.project_name.

Claude Code plugin reads env vars from ~/.claude/settings.json under the env block

Links

Contributing

Contributions are welcome — see CONTRIBUTING.md for development setup, the contribution process, and the CLA.

License

Apache 2.0

About

Trace coding harnesses like claude-code, cursor, and codex to Arize AX or Phoenix

Resources

License

Code of conduct

Contributing

Stars

23 stars

Watchers

2 watching

Forks

Packages

 
 
 

Contributors

Languages