Skip to content

fix: avoid FileTooBig when local usage scans large rollout files#18

Merged
Loongphy merged 9 commits intomainfrom
fix/local-list-filetoobig
Mar 19, 2026
Merged

fix: avoid FileTooBig when local usage scans large rollout files#18
Loongphy merged 9 commits intomainfrom
fix/local-list-filetoobig

Conversation

@Loongphy
Copy link
Copy Markdown
Owner

@Loongphy Loongphy commented Mar 19, 2026

Summary

  • switch local rollout scanning from whole-file reads to streaming line reads
  • avoid failing with FileTooBig when the newest rollout file exceeds 10 MB
  • add regression coverage for rollout files larger than 10 MB

Validation

  • zig test src/main.zig -lc --test-filter "scan latest usage"
  • HOME=/tmp/codex-auth-list-check zig build run -- list
  • HOME=/tmp/codex-auth-list-check zig build run -- config api disable
  • HOME=/tmp/codex-auth-list-check zig build run -- list

Related

Note

Fix FileTooBig error when scanning large rollout files for usage data

  • Rewrites scanFileForUsage in sessions.zig to stream the file line-by-line using a buffered reader instead of reading the entire file into memory, allowing rollout files larger than 10 MiB to be processed.
  • Adds a max_rollout_line_bytes constant (10 MiB) to cap individual line size; oversized lines are discarded rather than failing the whole scan.
  • Switches JSON parsing to typed structs (UsageEventLineJson and friends) with ignore_unknown_fields, and adds a looksLikeUsageEventLine pre-check to skip irrelevant lines cheaply.
  • Adds tests in sessions_test.zig covering files >10 MiB, missing trailing newlines, oversized lines, and windows without used_percent.

Macroscope summarized 2363a43.

@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new bot commented Mar 19, 2026

Open in StackBlitz

@loongphy/codex-auth-darwin-arm64

npx https://pkg.pr.new/@loongphy/codex-auth-darwin-arm64@18

@loongphy/codex-auth-darwin-x64

npx https://pkg.pr.new/@loongphy/codex-auth-darwin-x64@18

@loongphy/codex-auth-linux-x64

npx https://pkg.pr.new/@loongphy/codex-auth-linux-x64@18

@loongphy/codex-auth-win32-x64

npx https://pkg.pr.new/@loongphy/codex-auth-win32-x64@18

@loongphy/codex-auth

npx https://pkg.pr.new/@loongphy/codex-auth@18

commit: 2363a43

Add scripts/convert_tokens.sh to convert ~/.cli-proxy-api/token*.json
files into codex-auth-compatible auth.json format (auth_mode, tokens
with refresh_token, last_refresh). Document the workflow in README Q&A.
@Loongphy Loongphy merged commit 894c6c2 into main Mar 19, 2026
10 checks passed
@Loongphy Loongphy deleted the fix/local-list-filetoobig branch March 19, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

error: FileTooBig

1 participant