Commit 065f664
authored
perf(ccusage): unify agent adapter foundations (#1004)
* refactor(ccusage): move all-agent loading into adapters
* perf(ccusage): route codex reports through fast adapter
* perf(ccusage): parallelize codex adapter loading
* refactor(ccusage): route agent commands through adapters
* chore(ai): exhaustiveness check
* chore(ai): top-level fs-fixture is ok
* docs(ccusage): document adapter architecture
Record the adapter layering model before continuing the migration. The new architecture note makes source detection, file discovery, parsing, aggregation, worker boundaries, and parent-process row return explicit so future agents can be implemented on the same ccusage foundation instead of deprecated wrapper packages.
Also clarify testing guidance: prefer parser, loader, path, pricing, aggregation, and CLI output tests over schema-only assertions when realistic fixture logs already exercise validation behavior.
* docs(gunshi): preserve args inference
Document that Gunshi command factories should keep the concrete args type inferred from define(). Avoid Command<Args> and broad ReturnType<typeof define> annotations because they erase ctx.values option keys and value types.
* refactor(ccusage): organize agent adapters by source
Move agent-specific implementation under adapter/<agent>/ so the unified ccusage CLI owns the runtime logic instead of the deprecated wrapper packages.
Add shared adapter definition and log aggregation helpers, move Claude and Codex into directory adapters, split Amp and pi-agent into path/parser/schema/pricing files, and add worker-backed parsing for Amp and pi-agent JSON sources.
Replace new adapter path checks with an internal safe directory helper and add focused parser/path tests using fs-fixture. The affected adapter tests are green for shared, Amp, pi-agent, Codex, and agent command paths.
* refactor(ccusage): split opencode adapter sources
Separate OpenCode path discovery, schema, loader, pricing, and row aggregation so the adapter follows the shared detect/load/parse/aggregate layering.
OpenCode has multiple source kinds, so JSON message files now use the shared worker gating and SQLite remains a separate DB source. The loader still de-duplicates JSON messages against DB rows and keeps JSON+SQL totals combined.
Path checks now use the internal safe directory helper, pricing fetcher disposal uses using, and new fs-fixture tests cover path discovery, JSON message parsing, ignored non-billable messages, and row aggregation.
* fix(ccusage): require adapter pricing fetchers
* perf(ccusage): benchmark claude and codex fixtures
* test(ccusage): snapshot direct agent commands
* fix(ccusage): address adapter review findings
* fix(ccusage): align codex adapter totals
Use the Codex total_tokens field for all-agent adapter rows so ccusage all and ccusage codex report the same total token count. Cached input tokens are an input breakdown, and reasoning output tokens are informational because Codex output_tokens already carries the billable output value.
Add skipped local-data smoke tests for Claude and Codex adapter loaders. These run on developer machines with real log directories while staying inert on clean CI environments.
* refactor(ccusage): split codex adapter primitives
Move Codex adapter path discovery, pricing, usage arithmetic, and shared types out of the large adapter index. This makes the adapter boundary match the other agent adapters and keeps future Codex parser work localized.
Codex detection now short-circuits when it finds the first JSONL session file instead of collecting every session path just to answer a boolean. Focused Codex adapter tests and tsgo typecheck pass.
* fix(ccusage): group claude subagent session logs
Resolve Claude session report paths with the session id from each entry when available, so flat session files and nested subagent logs aggregate under the same session.
This matches the real Claude projects layout where both project/session.jsonl and project/session/subagents/*.jsonl can exist for one conversation.
* refactor(ccusage): split codex parser worker
Move Codex JSONL parsing and worker fan-out into adapter/codex/parser.ts so the public adapter index only handles report aggregation and pricing wiring.
The focused Codex adapter tests and package typecheck confirm the extracted parser keeps the same behavior.
* refactor(ccusage): move claude loader into adapter
Move the Claude-specific data loader under adapter/claude and update internal imports to use the adapter path directly.
No re-export shim is kept because the package only exports the CLI entrypoint and all remaining data-loader references are internal.
* docs(ccusage): document adapter migration workflow
Record the adapter migration checklist for future agents, including direct adapter imports, shared ccusage foundations, local smoke tests, cmux validation, and benchmark parity.
* refactor(ccusage): share indexed adapter workers
Move the repeated worker fan-out, file-size chunking, and indexed result restoration into @ccusage/internal so Amp, Codex, OpenCode, and pi-agent use the same worker foundation as the optimized ccusage loader path.
This also keeps Codex totals aligned when raw token_count payloads omit total_tokens by including reasoning_output_tokens in the parser fallback. The CLI output snapshot is updated for the corrected Codex total semantics.
* docs(ccusage): clarify adapter worker entry ownership
Record that adapter source logic belongs under apps/ccusage/src/adapter while src/data-loader.ts remains a dedicated bundled worker entry for the Claude data-loader chunk introduced by PR #984.
This keeps future adapter migrations from adding root-level source shims while preserving the build entry needed by the optimized loader.
* build(ccusage): bundle adapter offline pricing macros
Move the Amp and Codex macro imports into their pricing modules and include those modules in the tsdown macro transform. This mirrors the standalone packages and prevents offline mode from retaining the runtime prefetch path in the bundled ccusage CLI.
The adapter command modules now receive stable offline pricing loaders instead of importing macro functions directly.
* test(ccusage): skip local claude smoke test in ci
CI can have a Claude projects directory without local usage rows, which makes the real-data smoke test fail even though it is only meant for developer machines with actual logs.
Keep the smoke test enabled locally, but skip it whenever CI=true so the deterministic fixture tests remain the CI contract.
* docs(internal): document indexed worker collection
Add API documentation for the shared indexed worker helper requested during PR review. The comment records the ordering guarantee and the zero-worker fallback so adapter callers can reason about the shared worker path without reading the implementation.
* fix(ccusage): share usage loading progress
Move the all-agent loading spinner into a reusable progress component and wire it through all agent report commands. This keeps every direct agent command on the same single-spinner behavior instead of letting per-agent spinners or LiteLLM pricing logs compete with terminal rendering.
Route LiteLLM pricing messages through the progress component while a TTY spinner is active. JSON output continues to keep stdout machine-readable and the verification covers all, claude, codex, opencode, amp, and pi JSON modes.
* perf(ccusage): share adapter file primitives
Move whole-file text reads and cheap source detection into @ccusage/internal/fs so all coding-agent adapters use the same IO foundation.
Amp and OpenCode now read JSON message files through readTextFile(), Codex reads config.toml through the same helper, and JSONL buffering reuses readBufferedTextFile(). Claude keeps its byte scanner fast path while using the shared text fallback and transcript reader.
Detection for Claude, Codex, Amp, OpenCode, and pi-agent now uses hasFileRecursive() where only existence is needed, avoiding full file list materialization. The adapter architecture notes now document the shared optimization baseline.
* test(ccusage): allow local codex smoke test to finish
The local Codex smoke test reads real user sessions when CODEX_HOME exists. On machines with substantial Codex history it can exceed Vitest\x27s default 5 second timeout even though CI skips the test on clean runners.
Keep the smoke test in place and give it an explicit 30 second timeout so it remains useful for validating real local data without making the full suite fail spuriously.
* chore: minimise deprecated agent packages
Replace the standalone agent package implementations with dependency-free compatibility stubs that print use npx ccusage instead and exit non-zero.
The canonical implementation now lives in apps/ccusage adapters, so the deprecated @ccusage/amp, @ccusage/codex, @ccusage/opencode, and @ccusage/pi packages no longer need local source, build, lint, test, or TypeScript configuration files.
Removing those package-local dependencies shrinks the workspace and prevents future fixes from being split between ccusage and deprecated wrapper packages.
* perf(ccusage): bound agent fallback parsing
Add a shared mapWithConcurrency helper for non-worker file parsing paths. Worker-backed adapters already use chunked worker fan-out, but Amp and pi-agent still fell back to unbounded Promise.all when workers are unavailable, including during source-runtime execution and tests.
Use the shared helper in Amp thread parsing and pi-agent JSONL parsing so the fallback path keeps stable output order without opening every file at once. Document the bounded fallback requirement in the adapter architecture notes so future adapters follow the same baseline.
Validation: pnpm run format; pnpm typecheck; pnpm --filter @ccusage/internal test src/workers.ts; pnpm --filter ccusage test src/adapter/amp/parser.ts src/adapter/pi/parser.ts; pnpm --filter ccusage run build
* test(ccusage): allow local claude smoke test to finish
The optional local-data smoke test can exceed the default Vitest timeout on real Claude projects. The test should remain enabled on developer machines when data exists, so extend its timeout instead of skipping or weakening the assertion.
Validation: pnpm --filter ccusage test src/adapter/claude/index.ts; pnpm run format; pnpm run test; pnpm typecheck; pnpm --filter ccusage run build
* refactor(ccusage): reuse bounded fallback mapper
Claude kept a local mapWithConcurrency implementation while Amp and pi-agent now use the shared worker utility. Move Claude fallback parsing to the same internal helper so all adapter fallback file parsing paths use one bounded, order-preserving primitive.
This keeps the data-loader chunk behavior aligned with the adapter architecture while preserving the worker fast path and existing output shape.
Validation: pnpm run format; pnpm --filter ccusage test src/adapter/claude/data-loader.ts src/adapter/claude/index.ts; pnpm typecheck; pnpm --filter ccusage run build; pnpm run test
* fix(internal): reject empty worker exits
Reject worker collection promises when a worker exits successfully before posting results. Without this guard, a clean exit without a message left collectIndexedFileWorkerResults unresolved and could stall adapter loading indefinitely.
Add an in-source regression test that reproduces the missing-message exit path with a mocked worker.
Validation: pnpm --filter @ccusage/internal test src/workers.ts; pnpm run format; pnpm typecheck; pnpm run test; pnpm --filter ccusage run build
* fix(ccusage): honor merged session options
Use the merged configuration values when loading Claude session rows so config-file defaults and CLI overrides flow through the same path as the other commands.
Restore the logger level unconditionally after the loading lifecycle. JSON mode and progress mode both lower the global logger level, so the command should always put it back before returning or throwing.
Validation: pnpm run format; pnpm typecheck; pnpm run test; pnpm --filter ccusage run build
* refactor(ccusage): share codex aggregation pipeline
Extract the shared Codex event grouping and pricing lookup path so the all-agent rows and Codex-specific report rows use one aggregation implementation.
Keep output-shape mapping in the public loader functions while centralizing date filtering, model grouping, fallback-model tagging, pricing lookup, and owned pricing fetcher disposal.
Validation: pnpm --filter ccusage test src/adapter/codex/index.ts; pnpm run format; pnpm typecheck; pnpm run test; pnpm --filter ccusage run build
* docs(internal): document file helper contracts
Document the exported fs and JSONL helpers that are now shared by the agent adapters. The comments explain Bun-backed reads, buffered-read null behavior, recursive search tolerance, and JSONL line callback semantics without changing runtime behavior.
Validation: pnpm --filter @ccusage/internal test src/fs.ts src/jsonl.ts; pnpm run format; pnpm typecheck; pnpm run test; pnpm --filter ccusage run build
* docs(skills): mention AI reviewers in replies
Document that replies to AI reviewer inline comments should explicitly mention the bot handle. CodeRabbit can otherwise classify a direct fix reply as human discussion and skip follow-up review.
Use @coderabbitai for CodeRabbit and the current Cubic bot handle from the PR/check UI for Cubic replies.
Validation: pnpm typecheck
* fix(ccusage): detect all agents before json output
Compute the all-agent detection list before branching on output mode so JSON and table output load the same detected agent set.
This removes the fallback to resolveAllAgents from the all command load path. JSON mode no longer silently widens the load set beyond what detection found, while the banner remains gated to table output only.
Validation: pnpm --filter ccusage test src/commands/all.ts; pnpm typecheck; pnpm --filter ccusage test test/cli-output.test.ts; pnpm run test; pnpm --filter ccusage run build.
* docs: prefer unified agent commands
Update Codex, OpenCode, and pi-agent guide examples to use the canonical ccusage agent subcommands instead of the deprecated standalone wrapper packages.
The compatibility package notes remain where useful, but normal installation, alias, and command examples now point at ccusage codex, ccusage opencode, and ccusage pi. Bun examples omit @latest because bunx can run ccusage directly.
Validation: pnpm run format; pnpm --filter docs build.
* ci(ccusage): report fixture throughput
Add fixture byte and file counts to the ccusage performance PR comment so Claude and Codex timings can be interpreted with their input sizes.
Render per-command input size plus base and PR throughput columns. This keeps absolute median timing visible while making uneven Claude and Codex fixture sizes explicit in the same table.
Validation: pnpm --filter ccusage test test/perf-scripts.test.ts; pnpm run format; pnpm typecheck; pnpm --filter ccusage run build.
* ci(ccusage): use equal perf fixture defaults
Raise the Codex large fixture default from 256 MiB to 1024 MiB so CI compares Claude and Codex against the same generated input size unless explicitly overridden.
Move the performance script assertions into in-source Vitest blocks beside the script logic. The ccusage Vitest config now includes only these performance scripts as script in-source tests, avoiding unrelated Bun-only scripts that are not currently Vite-resolvable.
Validated with pnpm --filter ccusage test scripts/compare-pr-performance.ts scripts/generate-large-fixture.ts, pnpm run format, pnpm typecheck, pnpm --filter ccusage run build, and pnpm run test.
* test(ccusage): avoid dynamic fixture imports
Use the existing top-level fs-fixture import inside Claude data-loader in-source tests instead of awaiting dynamic imports inside describe blocks.
This keeps the tests aligned with the repository in-source Vitest guidance without changing loader behavior.
Validated with pnpm --filter ccusage test src/adapter/claude/data-loader.ts, pnpm run format, pnpm typecheck, and pnpm --filter ccusage run build.
* perf(ccusage): share JSONL marker scanning
Move the Claude byte-buffered JSONL marker scan into @ccusage/internal and make Claude, Codex, and pi-agent use the same primitive. This keeps JSONL adapters from decoding unrelated log lines while preserving the existing stream fallback for oversized files.
Codex also now sends worker events through typed-array transfer payloads instead of cloning large object arrays. The payload keeps timestamps, sessions, model indexes, numeric token fields, and fallback-model flags separate so worker transport matches the optimized Claude pattern more closely.
Validated with focused adapter tests, full format/typecheck/test, and JSON parity checks against HEAD for Claude, Codex, and pi-agent outputs.
* docs(ccusage): define shared adapter optimization baseline
Document that matching Claude means reusing the byte marker scanner for JSONL adapters and avoiding large worker object-array payloads when compact transfer payloads or worker-side aggregation are practical.
Also clarify that whole-JSON and SQLite adapters do not benefit from JSONL marker scanning directly, but should still share worker gating, result ordering, pricing lifecycle, output formatting, and compact payload helpers where their source shape allows it.
* refactor(ccusage): share adapter worker primitives
Move remaining Claude-local JSONL worker helpers onto the shared internal worker and JSONL primitives so the adapter no longer carries duplicate file-size chunking, worker response, or line-reader implementations.
Extend defineAgentLogLoader with one-time prepared state and async usage extraction. Amp and OpenCode now use the same row aggregation foundation as pi while keeping pricing fetchers scoped once per command and only calculating cost after date filtering.
Add shared indexed worker data and pricing context helpers so Codex, Amp, OpenCode, and pi workers use the same message/data shapes.
* docs(agents): add deprecated package readmes
Restore minimal README files for the deprecated standalone agent packages. Keep package-specific badges visible while replacing the old package documentation with a clear deprecation notice and the unified bunx ccusage command to use instead.
* fix(internal): preserve worker count os mocking
Use the node:os module object inside shared worker utilities so tests that spy on availableParallelism continue to affect Claude worker-count expectations after the worker-count helper moved into @ccusage/internal.
This keeps the refactor behavior deterministic across CI CPU counts without changing production worker-count logic.
* fix(ccusage): address adapter review edge cases
Fix timezone date/month cache keys so non-hour timezone boundaries cannot reuse a UTC-hour cache entry.
Decode buffered marker JSONL lines as UTF-8 and report marker indexes against the decoded string, preserving non-ASCII log content.
* fix(ccusage): harden adapter date keys
Reject impossible calendar date filters before they reach range filtering so invalid CLI input cannot silently produce wrong reports.
Build daily and monthly adapter keys from Intl formatToParts instead of parsing locale display strings. This keeps the existing timezone-aware Intl behavior while making machine-readable grouping keys independent of runtime date formatting patterns.
* perf(ccusage): keep adapter date key fast path
Avoid forcing formatToParts through the normal adapter date and month key path. The hot path now keeps the existing machine-key format result when Intl already returns YYYY-MM-DD or YYYY-MM, while retaining the formatToParts fallback for runtimes with different locale output.
UTC ISO timestamps also use direct string slices, which keeps the CI benchmark and default log path away from Intl allocation in tight loops. This preserves the review portability fix without penalizing Claude and Codex loader throughput.
* perf(ccusage): restore agent JSONL hot paths
Keep the shared JSONL marker helper while restoring the agent-specific hot paths that were lost during the scanner consolidation.
Claude now uses the shared helper single-marker byte scanner with latin1 decoding and byte marker indexes, matching the previous optimized loader behavior. Codex uses line scanning for marker-dense logs instead of the generic multi-marker candidate map.
Adapter date keys now keep the default timezone as the local system timezone without constructing Intl formatters on the hot path. Explicit --timezone values still use Intl when needed, while UTC uses direct UTC formatting.
Validation: pnpm run format; pnpm typecheck; pnpm run test; pnpm --filter ccusage build. Local parity against main through 2026-05-14 matched for claude/codex JSON in both TZ=UTC and default local timezone.
* perf(internal): preserve synchronous JSONL marker scans
Add an explicit synchronous callback mode to the shared JSONL marker helper so agent loaders that do purely synchronous parsing do not pay per-row Promise handling overhead.
Keep markerIndex byte semantics consistent for line-scan and text fallback paths by converting decoded-string indexes back to byte offsets with Buffer.byteLength. This addresses the latest CodeRabbit JSONL marker-index comment without failing large-file fallback paths.
Claude and Codex opt into the synchronous mode for their hot JSONL parser callbacks. Validation: pnpm run format; pnpm typecheck; pnpm --filter @ccusage/internal test src/jsonl.ts --run; pnpm --filter ccusage test src/adapter/codex/parser.ts --run; pnpm --filter ccusage build; JSON parity against main through 2026-05-14 for claude/codex with TZ=UTC.
* docs(internal): document JSONL marker scanning contract
Document the marker-processing options used by the shared JSONL scanner so callers can see the byte versus decoded index guarantees without reading the implementation.
This addresses the CodeRabbit documentation nit without changing scanner behavior or benchmark-sensitive code paths.1 parent 5280515 commit 065f664
163 files changed
Lines changed: 14909 additions & 17333 deletions
File tree
- .agents/skills
- ccusage-agent-sources
- ccusage-testing
- pr-ai-review-workflow
- typescript-style
- use-gunshi-cli
- .github/workflows
- apps
- amp
- src
- commands
- ccusage
- scripts
- src
- adapter
- amp
- claude
- codex
- opencode
- pi
- commands
- test
- fixtures/codex/sessions/project-alpha
- snapshots/cli-output
- codex
- src
- commands
- opencode
- src
- commands
- pi
- src
- commands
- docs/guide
- codex
- opencode
- pi
- packages/internal/src
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
17 | 18 | | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
18 | 23 | | |
19 | 24 | | |
20 | 25 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
| 41 | + | |
40 | 42 | | |
41 | 43 | | |
42 | 44 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
65 | 84 | | |
66 | 85 | | |
67 | 86 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| 44 | + | |
44 | 45 | | |
45 | 46 | | |
46 | 47 | | |
| |||
49 | 50 | | |
50 | 51 | | |
51 | 52 | | |
| 53 | + | |
52 | 54 | | |
53 | 55 | | |
54 | 56 | | |
| 57 | + | |
55 | 58 | | |
56 | 59 | | |
57 | 60 | | |
| |||
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
0 commit comments