Skip to content

fix(antigravity): restore local spend from all conversation stores and price every logged model - #1206

Merged
robinebers merged 13 commits into
robinebers:mainfrom
Nabsku:fix/antigravity-usage-logs-and-wal-v2
Sep 5, 2026
Merged

fix(antigravity): restore local spend from all conversation stores and price every logged model#1206
robinebers merged 13 commits into
robinebers:mainfrom
Nabsku:fix/antigravity-usage-logs-and-wal-v2

Conversation

@Nabsku

@Nabsku Nabsku commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Approved issue

Fixes #1202
Fixes #1215

TL;DR

Antigravity's local spend tile was empty or partial for most users. This PR makes the conversation-log scanner decode newer records, open WAL-mode databases, read every Antigravity surface's store, and price the placeholder and tiered model IDs Antigravity logs, grouped by model family.

What was happening

  • Newer agy versions dropped the embedded wall-clock timestamp from gen_metadata.data, so every generation record failed to decode and the tile showed nothing.
  • WAL-mode databases whose -shm sidecar cannot be opened read-only fail under sqlite3 -readonly with "unable to open database file (14)", so those conversations were skipped with a warning.
  • Only ~/.gemini/antigravity-cli/conversations was scanned. The Antigravity IDE, the Antigravity 2.0 app, and ACP sessions each keep their own store, so their usage was never counted.
  • With the model picker on its default, Antigravity logs gemini-default / gemini-pro-default as the model ID and the served model only as a display label ("Gemini 3.1 Pro (High)"). No alias rule matched either, so about 12% of records were unpriced and the tile showed "Unknown models found".
  • Prompt-context bookkeeping rows with no model and only a system-prompt token count decoded as generations and surfaced as "Unknown Antigravity Model".
  • The breakdown listed every logged ID as its own row (gemini-3.1-pro-low, gemini-3.7-flash-tiered, gemini-3.7-flash-exp-a) even though they bill at the base model's rate.

What this changes

  • Timestamps: the decoder takes the correlated steps.metadata blob and uses its Timestamp when the embedded timing is absent. Records with neither are dropped rather than attributed to the database's modification day. The scanner probes for the steps.metadata column per database instead of matching error text, and re-probes a store that lacked it so one that gains step timestamps is re-read without a restart.
  • WAL reads: SQLiteCLIAccessor retries any failed -readonly open with PRAGMA query_only = ON, so the fallback connection still cannot write.
  • Stores: the scanner discovers every ~/.gemini/antigravity*/conversations directory, so a new Antigravity surface needs no release. Symlinked or repeated stores are scanned once. An unreadable root, or a ~/.gemini that cannot be listed, is reported through the read-failure path instead of aborting the scan or reading as no usage.
  • Model resolution: the decoder exposes both the internal ID and the display label. The scanner prices placeholder rows by the label first and falls back to the ID, folds -tiered into the base model, and only reports a row as unknown when nothing prices. Bookkeeping rows produce no event.
  • Breakdown rows group by canonical pricing family, as Cursor's do, with the catalog's -preview suffix dropped: effort variants, display labels, placeholder IDs, and experiment slots all count under one gemini-3.1-pro or gemini-3.7-flash row. Names no alias rule knows keep their raw text.
  • Alias rules: display labels with any parenthetical qualifier for Gemini 3.1 Pro, 3.5 to 3.8 Flash, and Claude Opus/Sonnet 4.6; gemini-pro-default and gemini-pro-agent as Gemini 3.1 Pro; Flash -exp-[a-z] experiment slots; the claude-opus-4-6 / claude-sonnet-4-6 slug spellings. gemini-default is deliberately unaliased because its label has pointed at different Flash generations over time.
  • Docs: docs/providers/antigravity.md and docs/pricing.md describe the stores, placeholder handling, tiered IDs, and row grouping.

Heads-up

  • Users who ran the IDE or the 2.0 app will see Antigravity spend jump on the first refresh after this ships, because that history was never counted.
  • Grouping by family hides the Low/High effort split. Cursor keeps that as per-row variants; the accumulator Antigravity shares with Claude and Codex has no variant support, so that is a follow-up if anyone misses it.
  • The query-only retry launches a second sqlite3 process per batch on affected databases. Caching the working mode per path is a follow-up.

Tests

  • Verified against 5,166 records across all four local stores on one machine. The IDE and 2.0 databases reproduce the WAL open failure and read correctly through the retry. After the change the unknown-models warning is empty, 30-day spend prices in full, and the breakdown shows six family rows.
  • Antigravity tests are split by concern (AntigravityProtoDecoderTests, AntigravityDbSchemaTests, AntigravityDbUsageScannerTests, shared fixtures in AntigravityTestSupport). They cover step-metadata timestamps, the no-timestamp drop, a real WAL-mode database, a steps table without metadata, a store that gains step timestamps, store discovery and symlink de-duplication, an unlistable ~/.gemini, placeholder and tiered resolution, the priced-ID fallback, family grouping, and bookkeeping rows. ModelPricingTests covers the new alias rules; CredentialCacheIntegrityTests pins the retry arguments.

Screenshots

Not applicable (data ingestion for existing spend tiles and charts).

- Make timing field 9.4 optional in AntigravityProtoDecoder, falling back to steps.metadata timestamp or DB modification time
- Query steps.metadata in AntigravityDbUsageScanner with fallback to legacy query when steps table is absent
- Fall back to non-readonly open in SQLiteCLIAccessor when WAL mode errors with SQLITE_CANTOPEN (14)
- Add unit tests for step metadata timestamp extraction and fallback handling
- Fixes robinebers#1202
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Thanks for your interest in contributing to OpenUsage!

External pull requests must reference an open issue that:

  1. Has been approved by a maintainer with the approved label.
  2. Is assigned to the pull request author (@Nabsku).

Please discuss the change on an issue first, wait for a maintainer to approve and assign it to you, then reopen this pull request with Fixes #123 in its description.

Read the contribution guidelines.

@validatedev

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: efdafe8965

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/OpenUsage/Providers/Antigravity/AntigravityDbUsageScanner.swift Outdated
…a errors

- Match SQLiteError.queryFailed stderr for missing table/column before falling back to legacyDataSQL
- Safely rethrow transient errors (process timeouts, database locks, I/O errors)
- Add unit tests verifying schema-missing error discrimination

@robinebers robinebers left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review — Codex

AI-generated review, posted on behalf of @robinebers.

Requesting changes on b10efd0 for two findings detailed inline: the file-modification-time fallback makes daily usage change after a restart, and the SQL change breaks an existing regression test.

Validation: the code builds, and 75 of 76 selected Antigravity and credential-system-client tests pass. The failing test has two assertion failures. Additional temporary tests against real SQLite databases confirm that the modern step-timestamp path works and reproduce the restart-dependent date shift when timestamps are absent. The temporary tests were removed after review.

The PR remains unmerged.

Comment thread Sources/OpenUsage/Providers/Antigravity/AntigravityDbUsageScanner.swift Outdated
Comment thread Sources/OpenUsage/Providers/Antigravity/AntigravityDbUsageScanner.swift Outdated
@validatedev validatedev self-assigned this Sep 4, 2026
… SQL bounds test

Events with no embedded timing and no steps.metadata timestamp were stamped
with the database modification time, which changes on every write, so the
same usage moved to a different day after a restart. The step timestamp is
now the only fallback; events with neither are skipped.

Update the batch SQL test for the aliased g.data columns, cover the
steps.metadata and legacy query bounds, and add a real-SQLite regression
test that advances the file's modification time between scans.
…to the decoder, query-only WAL retry

- One dataSQL builder with includeSteps instead of two near-identical queries
- Replace stderr matching for missing steps with a cached sqlite_master probe per database
- generationEvent takes the steps.metadata blob; shared Timestamp reader for both paths
- WAL fallback opens with PRAGMA query_only instead of a write-capable connection
- Test helpers for hex encoding and step metadata; cover the query-only retry
… the CLI

The IDE, the Antigravity 2.0 app, and ACP sessions each keep their own
conversations directory under ~/.gemini, so usage from those surfaces was
never counted. Scan all four roots; a missing or unreadable root is reported
per directory instead of aborting the whole scan.
@github-actions github-actions Bot added the docs label Sep 4, 2026
Antigravity resolves subagent tiers (flash_lite, flash, pro) through a
server-sent TieredModelConfig and logs the result as e.g.
gemini-3.7-flash-tiered. It is the same model at the same rate, so price it
through the Gemini alias rules and show it under the base name in the
breakdown instead of as a separate row.
…lay label and skip bookkeeping rows

gen_metadata rows written with the picker on its default carry gemini-default
or gemini-pro-default in field 19 and the served model as a label in field 21,
so price and list those turns by the label. Rows with neither ID nor label that
carry only a system-prompt count are prompt-context records, not generations,
and no longer surface as an unknown model. Alias rules cover the labels with
effort levels, Antigravity's internal Gemini 3.1 Pro IDs, and Flash experiment
slots. Fixes robinebers#1215.
…versation stores, retry any read-only open failure

The decoder now exposes the internal model ID and display label instead of
choosing between them; the scanner prices placeholder rows by label first and
falls back to the ID, and folds -tiered there too, so the alias rules no
longer duplicate that suffix. Display-label alias rules share one qualifier
form. Conversation stores are discovered under ~/.gemini/antigravity*/ instead
of a fixed list. The SQLite accessor retries with query_only on any failed
read-only open rather than matching sqlite3's error text.
Effort variants, display labels, placeholder IDs, and experiment slots bill
at the base model's rate, so they now share one row named by the canonical
pricing family without the catalog's -preview suffix. Adds the
claude-opus-4-6 / claude-sonnet-4-6 slug aliases so those rows fold too.
@validatedev validatedev changed the title fix(antigravity): restore usage log decoding and WAL-mode database reads fix(antigravity): restore local spend from all conversation stores and price every logged model Sep 4, 2026
Probe steps.metadata as a column, not just the table, and re-probe while a
database lacks it so a store that gains step timestamps is re-read without a
restart. De-duplicate symlinked or repeated stores so conversations are not
counted twice. Surface a failed ~/.gemini listing through the read-failure
reporter instead of reading it as no usage. Add a real WAL-mode fixture test
and split the Antigravity test file by concern.
…d Cursor

ModelPricing gains a memoized canonicalName(for:) and familyName(for:stripping:)
so the Antigravity and Cursor breakdowns fold rows through one helper instead
of two private copies, and the alias scan runs once per distinct name. The
Antigravity scanner lists directories through one helper, discovers and
de-duplicates database files in one step, and owns the unknown-model label.
Antigravity test fixtures move to the shared support file.
@validatedev

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 94e5a4b7fb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/OpenUsage/Providers/Antigravity/AntigravityDbUsageScanner.swift Outdated
…ipping them

An existence check dropped a store whose conversations directory could not be
stat'ed before the listing ran, so a permission or I/O problem read as no usage.
Every candidate store now goes through the listing, which treats a missing
directory as empty and reports one it cannot read.
@robinebers
robinebers merged commit 60f29f9 into robinebers:main Sep 5, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

3 participants