feat(observability): capture every miner AI generation, and stop fabricating zero token counts

[JSONbored]

Summary

Two AI-observability defects in one place: generations that were never reported, and token counts that were reported when nothing had been measured.

1. Every miner AI generation is now captured (#10200)

withCodingAgentAiGenerationCapture was attached at one of the two driver construction sites — inside constructProductionCodingAgentDriver. runCodingAgentAttempt builds its driver through createCodingAgentDriver directly, so it never passed through that site and every attempt it ran was uncaptured, silently. runChatGrounding drives a full query() session with a real turn budget and was never instrumented at all — it read only the assistant/user messages it turns into wire events, so the SDK's own result frame, carrying the session's usage and cost, was on the wire and thrown away.

Both are the shape #10170 catalogues: two or more places that must agree, with nothing enforcing it. Fixed the way #10127 fixed its own instance — make the bypass unrepresentable rather than document it:

• A host-registered sink (packages/loopover-engine/src/miner/ai-generation-sink.ts) keeps posthog-node out of the portable engine package while letting the engine emit. initMinerPostHog registers it only on the opt-in path, so the no-phone-home default (docs(selfhost): add combined ORB + AMS quickstart #6011) holds by construction.
• The capture moved into createCodingAgentDriver, the one chokepoint every real driver is built through. A future third construction site cannot opt out, because there is nothing left for it to forget to attach.
• runChatGrounding now folds the result frame and reports model/provider/latency plus real tokens and cost, on both the completed and the thrown path.

noop, injected test drivers, and dry-run/paused attempts are deliberately not captured: none reaches a model, so an event there would fabricate a generation that never happened.

2. No fabricated zero token counts (#10207)

capturePostHogAiGeneration (ORB) and captureMinerPostHogAiGeneration (miner) both did:

$ai_input_tokens: Number.isFinite(event.inputTokens) ? event.inputTokens : 0,

so a provider that reported nothing contributed a 0 PostHog cannot tell apart from a measured one — dragging every tokens-per-call and input:output ratio toward zero. $ai_total_cost_usd alongside them has always been conditional for exactly this reason; these two were the outliers. Now omitted when absent, and a genuinely measured 0 still lands, because absence is what is tested, not falsiness.

Verified against the Orb rather than assumed, which corrects this issue's own open question. #10207 listed two candidate mechanisms and asked which applied. ai_usage_events on edge-nl-01 over 24h shows real per-call splits for the providers that carry the volume:

provider	model	calls	with input	with output
ollama	bge-m3	1535	1535	0 (embeddings have none)
claude-code	claude-sonnet-5	256	256	256
ollama	qwen3:8b	312	312	312

So mechanism 1 ("providers genuinely report none") is false for those — the ORB had the numbers all along, and the PostHog nulls were the scrubber (#10211/#10212). The fabricated zeros were coming from the handful that really do report nothing (the Workers AI rows), and were being averaged in as if they were measurements. That is what this half fixes.

Deliberate deviation from one deliverable

#10200 asks that a chat-grounding provider-config error be visible as telemetry. It is not emitted here: no model is ever reached on that path, so an $ai_generation would fabricate spend — the same rule the other half of this PR enforces. That case is a request that produced no generation, which the ORB side gives its own separate selfhost_ai_degraded event (#10186); the miner has no counterpart yet. Called out rather than quietly dropped, and pinned by a test.

Also in this PR

One commit regenerating packages/loopover-contract/src/api-schemas.ts. #10160 added linkedIssueMaintainerExempt without regenerating it, so contract:api-schemas:check has been failing on main since — it is in the local test:ci aggregate but not wired into ci.yml, which is why nothing surfaced it.

Closes #10200
Closes #10207

Scope

• The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
• This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
• This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
• I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

• git diff --check
• npm run actionlint
• npm run typecheck
• npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed
• npm run test:workers
• npm run build:mcp
• npm run test:mcp-pack
• npm run ui:openapi:check
• npm run ui:lint
• npm run ui:typecheck
• npm run ui:build
• npm audit --audit-level=moderate
• New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

Detail on what was run:

• Full unsharded suite post-rebase: 26,573 passed, 0 failed. @loopover/engine's own node:test suite: 955 passed, 0 failed.
• Patch coverage measured line-by-line against lcov, not read off a summary. Every changed file is 100% lines and 100% branches: ai-generation-sink.ts (13/13, 4/4), driver-factory.ts (57/57, 61/61), chat-grounding.ts (89/89, 91/91), agent-sdk-driver.ts (53/53, 64/64), loopover-miner/lib/posthog.ts (42/42, 45/45), coding-agent-construction.ts (23/23, 15/15). src/selfhost/posthog.ts is 174/174 lines with one uncovered branch at line 95 — pre-existing and outside this diff.
• Both new guards were mutation-tested: removing the factory capture fails the runCodingAgentAttempt regression test; capturing noop fails its own test.
• Engine changes are covered from both upload sides — test/unit/** (vitest) and packages/loopover-engine/test/** (node:test) — since only the second behavior-tests the package.
• Also green: test:engine-parity, test:driver-parity, test:live-gate-parity, test:miner-pack, build:miner, and the full drift sweep (db:migrations:check, db:schema-drift:check, selfhost:env-reference:check, miner:env-reference:check, cf-typegen:check, docs:drift-check, coverage-boltons:check, dead-exports:check, manifest:drift-check, engine-parity:drift-check, contract:api-schemas:check).
• Unchecked boxes: no UI file is touched, so the UI toolchain is left to CI. npm audit reports 13 pre-existing advisories, all transitive under release-please/code-suggester and unchanged by this PR — the diff adds zero dependencies (package.json/package-lock.json untouched).

Safety

• No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
• Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
• Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
• API/OpenAPI/MCP behavior is updated and tested where needed.
• UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
• Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails.
• Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

Metadata-only, unchanged. No prompt, transcript, diff or tool output reaches any event added here — the chat-grounding capture reads the result frame's usage and cost and nothing else, and the result frame is never re-emitted as a wire event. The privacy boundary this endpoint enforces (tool allowlist, PUBLIC_FIELD_BLOCKLIST redaction on text and tool output) is untouched. The opt-in posture is likewise unchanged: no sink is registered unless the operator set their own PostHog key.

UI Evidence

Not applicable — no visible UI, frontend, docs, or extension change.

Notes

runChatGrounding and runCodingAgentAttempt have no production call site today — chat grounding is built for the miner-ui chat rail (#6517, still epic work under #9184/#9191), and the iterate loop consumes a driver from constructProductionCodingAgentDriver instead. So this fixes latent blindness, not a live gap in current spend reporting. The value is structural: the capture is now attached where it cannot be bypassed, so those surfaces are already instrumented when they are wired, rather than shipping dark and being noticed later.

[loopover-orb]

Important

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏳ LoopOver is waiting…

LoopOver has seen this pull request and is waiting on CI checks to finish before reviewing it. This comment will update once the review runs.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed · 🟨 Waiting}

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	loopover-ui	26afd39	Commit Preview URL Branch Preview URL	Jul 31 2026, 01:31 PM

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

❌ Patch coverage is 99.61977% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 92.25%. Comparing base (6fdbbc1) to head (26afd39).
⚠️ Report is 10 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...ckages/loopover-engine/src/miner/chat-grounding.ts	98.38%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10233      +/-   ##
==========================================
+ Coverage   92.22%   92.25%   +0.02%     
==========================================
  Files         937      938       +1     
  Lines      114413   114642     +229     
  Branches    27662    27680      +18     
==========================================
+ Hits       105520   105758     +238     
+ Misses       7588     7579       -9     
  Partials     1305     1305              

Flag	Coverage Δ
backend	95.67% <100.00%> (+<0.01%)	⬆️
engine	74.05% <98.43%> (+0.19%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
packages/loopover-contract/src/api-schemas.ts	100.00% <ø> (ø)
packages/loopover-engine/src/index.ts	100.00% <100.00%> (ø)
...ages/loopover-engine/src/miner/agent-sdk-driver.ts	95.81% <100.00%> (+0.23%)	⬆️
...es/loopover-engine/src/miner/ai-generation-sink.ts	100.00% <100.00%> (ø)
...ckages/loopover-engine/src/miner/driver-factory.ts	97.87% <100.00%> (+3.73%)	⬆️
...es/loopover-miner/lib/coding-agent-construction.ts	100.00% <100.00%> (ø)
packages/loopover-miner/lib/posthog.ts	100.00% <100.00%> (ø)
src/selfhost/posthog.ts	99.50% <100.00%> (ø)
...ckages/loopover-engine/src/miner/chat-grounding.ts	99.44% <98.38%> (-0.23%)	⬇️