Skip to content

scan-components: no per-component progress or cost in headless CI runs #803

Description

@s-tokutake

Environment

  • @openai/codex-security 0.1.25 (same behavior on 0.1.24)
  • GitHub Actions ubuntu-latest, Node 22.13.0, non-interactive (CI=true, no TTY)
  • Command shape:
codex-security scan-components <repo> \
  --components-file <plan.json> --output-dir <dir> \
  --headless --full-output --format json \
  --model gpt-5.6-sol --effort high --workers 1 --max-cost <n>

Summary

scan --headless prints periodic progress lines carrying phase, file counts, token counts, and a running cost estimate. scan-components --headless prints only component lifecycle transitions. During a long multi-component run there is no phase, no file coverage, no token counts, and no cost until the whole run ends.

Reproduction

  1. Plan components for a repository (scan-components <repo> --auto --plan-only --headless).
  2. Run scan-components headless in CI with the resulting components.json.
  3. Watch the job log.

Actual behavior

The log holds one line per component transition and nothing else:

codex-security: Web application started
(… hours of silence …)
codex-security: Matching component findings by root cause...
Component scans: 12 complete, 0 incomplete, 0 failed.

In our case one component held ~96% of a ~3,200-file repository, so a single started line was followed by silence for the whole run. Compare scan --headless on the same host, which prints:

[00:30] Running scan: preflight (committed changes) | Files: 0/211 | Tokens: 138,555 input, 102,316 cached, 1,922 output | Cost: $0.335

Expected behavior

Per-component progress and running cost in headless mode, comparable to what scan --headless already provides.

Where it comes from (0.1.25 dist)

  • dist/cli.js:2504 — the scan-event observer is wired to the dashboard or to nothing:

    onScanEvent: dashboard === null
        ? undefined
        : (event) => dashboard?.recordComponentEvent(event),
  • dist/component-scan.js — the per-component observers (onProgress, onActivity, onSessionEvent, onCost, onWorkerStatus, onWarning) are constructed only when options.onScanEvent !== undefined, so each security.run(...) runs with no observers at all in headless mode.

  • dist/cli.js:2457-2461 — the dashboard is gated on !options.headless && !options.planOnly && errorOutput.isTTY === true && environment["CI"] === undefined && environment["TERM"] !== "dumb", so a CI runner can never take the dashboard path.

  • dist/cli.js:2515-2519onProgress is the only non-dashboard writer, and it reports component status transitions only.

Net: the only surface that consumes cost and progress events for component scans is the one surface CI cannot use.

Impact

  • No way to distinguish a healthy long request from a hang, which is the concern in Show truthful Standard scan progress and completed review activity #70 but for scan-components.
  • No way to observe spend against --max-cost while it accrues. The per-component cost reaches summary.json only when the run completes, so a run cancelled midway leaves no cost record at all even though the spend happened.

Possible fix

When dashboard === null, wire onScanEvent to a line writer rather than leaving it undefined — at minimum forwarding cost and progress events prefixed with the component name, rate-limited the way scan --headless already does:

codex-security: Web application | reviewing files 18/211 | tokens 7,253,536 in / 63,967 out | cost $7.69

An explicit --progress flag would work equally well if silence is the intended default for headless.

Happy to test a patch against a real multi-component run.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions