Skip to content

Codex tribunal does not emit reviewer token usage into lens-completed events #311

Description

@SUaDtL

Summary

A Codex-hosted $ca-tribunal run completed every reviewer lens without emitting the optional tokens field in any lens-completed event. The generated telemetry therefore records tokens_actual: null, so the estimator cannot self-calibrate for Codex runs.

Observed

  • codeArbiter Codex plugin: 0.3.0
  • Host and orchestrator model: Codex / gpt-5-codex
  • Completed lenses: 11
  • lens-completed events containing tokens: 0 of 11
  • Estimated tokens: 3015018
  • Actual tokens: null
  • Public scrubbed reproduction: run-metrics 2026-07-15T09:05:10.9956267+00:00 #310

The tribunal run exposed each lens model, surface_seen, and finding count, but no token-usage value. No repository identity, paths, code, or finding text are needed to reproduce the reporting gap.

Expected

When Codex exposes reviewer usage, the host integration should record it on each lens-completed event so tokens_actual can sum the observed values. If the Codex agent surface does not expose usage, the run should record an explicit unavailability reason or capability state instead of leaving the cause implicit in a null value.

Impact

Codex tribunal estimates cannot be calibrated against measured consumption. Maintainers also cannot distinguish an unsupported host surface from a transient instrumentation failure by inspecting telemetry alone.

Acceptance criteria

  • Codex tribunal runs record numeric reviewer token usage when the host exposes it.
  • Telemetry sums the recorded values into tokens_actual.
  • When usage is unavailable, run state distinguishes an unsupported host capability from missing or failed instrumentation.
  • Automated coverage exercises the Codex unavailable-usage path and the numeric aggregation path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcodexOpenAI Codex CLI multi-host support (ADR-0011); code on feat/codex-support-m0telemetryOpt-in scrubbed tribunal run metrics

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions